Commands to delete excess backup videos and reorder versions

[susanodd]

Goal: from a selected set of GlossVideo objects in the admin, delete the video file and the object if the version is a backup.
Reorder the remaining backup videos for the glosses in the set.

@vanlummelhuizen if you're working today on Signbank, the new "video admin" command merely prints what it is doing, it does not actually change anything.
I added a rename to it to get the correct video file extension on the backup file.

Perhaps the command is a bit weird now. Or perhaps it should be split into multiple commands.

The command needs to operate on a selection of GlossVideo objects.
So it does what it's supposed to (although commented out and doesn't), and for the "remaining" objects that were not deleted, it renames them and fixes the version number.

Since the command currently "doesn't do anything other than print stuff" it is harmless to try on the real server.

[vanlummelhuizen]

@susanodd Thank you for this PR and making it not too big. I can see it does not do anything, but I assume it should do something in the future. I guess you first want to see that it is outputting the correct stuff before letting it (re)move and rename files.

Although the PR itself is not big, the functions you wrote are a bit long and sometimes hard to follow. I think there are several places where you could improve this. In function remove_backups:

• You can get a distinct set of glosses with distinct_glosses = Gloss.objects.filter(glossvideo__in=queryset).distinct() instead of

    glosses_in_queryset = [obj.gloss for obj in queryset]
    distinct_glosses = list(set(glosses_in_queryset))

Also, move that statement to the place where it is used.

• You could get rid of the if obj.version == 0: clause if you do queryset = queryset.filter(version__gt=0).

• This code:

    lookup_backup_files = dict()
    for gloss in distinct_glosses:
        lookup_backup_files[gloss] = GlossVideo.objects.filter(gloss=gloss, version__gt=0).order_by('version', 'id')
    for gloss, videos in lookup_backup_files.items():

could be reduced to

    for gloss in distinct_glosses:
        videos = GlossVideo.objects.filter(gloss=gloss, version__gt=0).order_by('version', 'id')

• There are ways of reducing the if DEBUG_VIDEOS: clauses to one line if there is only a print statement. E.g something like print('bla') if DEBUG_VIDEOS else None.

• Because there are two steps, move each step to a separate function.

[susanodd]

The improvements are much appreciated!

That the code seems in the wrong place has to do with it being evolved several times. I wanted to show the non-backup videos as well in the print statements to look at them as well. The glosses need to be retrieved from the selected GlossVideo objects before anything is deleted.

I will split it into several commands, functions. I'm trying to ascertain how to make it the most useful to clean up the objects.
The admin facilities can either do commands on selected objects or queries / filters. I can try making a filter on the type of video file.

The API does not check the video type. That is done in the browser when you select a file. I wanted to check the webm video type but I can't upload it on Safari. But for some reason, there are webm videos from the API. I had to copy locally a webm file onto an already existing backup file in order to check that it can change the name correctly. (webm can't play on MacOS nor on Ubuntu.)

Probably we need some conversion commands, too, in the admin. It is much easier to navigate to files in the admin than on Ubuntu.

[vanlummelhuizen]

The glosses need to be retrieved from the selected GlossVideo objects before anything is deleted.

You are right.

[susanodd]

@vanlummelhuizen I added a column to the GlossVideo table in the admin. It shows the "file type" (that returned by "file".)

Lots of the "mp4" files are actually Apple iTunes Video (.M4V) Video. So they should be ".mp4" or "m4v"?

The conversion "ensure_mp4" was checking for h264 for a very long time and just making them be mp4.

Can this ever be a problem? Or should we adhere to the extension belonging to the type of video?
(I did not do this, it's very old files that are on my local computer. They are all mp4 but most are actually m4v, from Apple.)
Firefox does not care and just plays them all. The only videos it can't play are webm.

[susanodd]

Regarding the requests for code improvement

• There are ways of reducing the if DEBUG_VIDEOS: clauses to one line if there is only a print statement. E.g something like print('bla') if DEBUG_VIDEOS else None.

I prefer not to do this, since it makes it difficult to skip over lines because of lack of indentation.

The other changes I have made.