Nemo 4.4.2 thumbnailing performance

[smurphos]

 * Nemo version 4.4.2
 * Is issue with desktop or windowed nemo? Window
 * Distribution - Mint 19.3

The test machine

steve@steve-Inspiron-5580:~$ inxi -SDMGCmxxxz
System:    Host: steve-Inspiron-5580 Kernel: 5.0.0-37-generic x86_64 bits: 64 compiler: gcc v: 7.4.0 
           Desktop: Cinnamon 4.4.6 wm: muffin 4.4.2 dm: LightDM 1.26.0 Distro: Linux Mint 19.3 Tricia 
           base: Ubuntu 18.04 bionic 
Machine:   Type: Laptop System: Dell product: Inspiron 5580 v: N/A serial: <filter> Chassis: type: 10 
           serial: <filter> 
           Mobo: Dell model: 0K0DFT v: A00 serial: <filter> UEFI: Dell v: 2.4.0 date: 07/02/2019 
Memory:    RAM: total: 7.51 GiB used: 1.19 GiB (15.9%) 
           RAM Report: permissions: Unable to run dmidecode. Root privileges required. 
CPU:       Topology: Quad Core model: Intel Core i5-8265U bits: 64 type: MT MCP arch: Kaby Lake rev: B 
           L2 cache: 6144 KiB 
           flags: lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 28800 
           Speed: 800 MHz min/max: 400/3900 MHz Core speeds (MHz): 1: 800 2: 800 3: 800 4: 800 5: 801 6: 800 7: 800 
           8: 800 
Graphics:  Device-1: Intel vendor: Dell driver: i915 v: kernel bus ID: 00:02.0 chip ID: 8086:3ea0 
           Display: x11 server: X.Org 1.20.4 driver: modesetting unloaded: fbdev,vesa resolution: 1920x1080~60Hz 
           OpenGL: renderer: Mesa DRI Intel UHD Graphics (Whiskey Lake 3x8 GT2) v: 4.5 Mesa 19.0.8 compat-v: 3.0 
           direct render: Yes 
Drives:    Local Storage: total: 1.14 TiB used: 882.34 GiB (75.4%) 
           ID-1: /dev/nvme0n1 vendor: Toshiba model: KBG40ZNS256G NVMe 256GB size: 238.47 GiB speed: 31.6 Gb/s 
           lanes: 4 serial: <filter> rev: 10410101 scheme: GPT 
           ID-2: /dev/sda type: USB vendor: Seagate model: Expansion size: 931.51 GiB serial: <filter> rev: 060E 
           scheme: MBR 

Issue

I reported during the beta Nemo being killed by the OOM controller whilst thumbnailing a folder of 21K image symlinks on a limited ram (2GB) virtual machine and have noted thumbnailing related performance issues for sometime. Several open issues regarding nemo peformance in general I think are related to thumbnailing and also memory use of nemo when navigating folders which large numbers of already thumbnailed images. I thought it might be helpful to try and benchmark some behaviour. These tests are done on the host machine specs above. The folders are on the SSD.

Test 1 - Thumbnail generation on a folder of 1000 jpg images in icon view at maximum zoom

Directory info

steve@steve-Inspiron-5580:~$ gio info /home/steve/Nemo_Test/1000_pics | grep metadata
  metadata::nemo-default-view: OAFIID:Nemo_File_Manager_Icon_View
  metadata::nemo-icon-view-zoom-level: 6
steve@steve-Inspiron-5580:~$ du --si ~/Nemo_Test/1000_pics
3.0G	/home/steve/Nemo_Test/1000_pics

Baseline ps aux output for nemo after running nemo -q; nemo

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    30633  6.9  1.0 1012356 82188 pts/0   Sl+  10:07   0:01 nemo

Then quit nemo, cleared the thumbnail cache and restarted nemo directly on the target folder.

steve@steve-Inspiron-5580:~$ nemo -q
steve@steve-Inspiron-5580:~$ rm -r ~/.cache/thumbnails
steve@steve-Inspiron-5580:~$ NEMO_BENCHMARK_LOADING=1 nemo /home/steve/Nemo_Test/1000_pics

** (nemo:28710): WARNING **: 10:12:55.113: Current gtk theme is not known to have nemo support (Cinnamox-Rhino) - checking...
Initializing nemo-image-converter extension
Nemo startup time: 1.133658 seconds
Folder load time: 0.396640 seconds
Idle...Folder load time: 0.567926 seconds

Nemo remained responsive and allowed me to scroll up and down the folder a little during thumbnailing.

ps aux output for nemo having thumbnailed all pics

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    28710 82.2  8.1 1547796 642060 pts/0  Sl+  10:12   1:43 nemo /home/steve/Nemo_Test/1000_pics

ps aux output for nemo having navigated back to parent folder

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    28710 31.7  1.8 1013740 147792 pts/0  Sl+  10:12   1:46 nemo /home/steve/Nemo_Test/1000_pics

ps aux output for nemo displaying the thumbnailed folder again.

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    28710 29.6  5.5 1310380 439148 pts/0  Rl+  10:12   1:57 nemo /home/steve/Nemo_Test/1000_pics

Load times on 2nd view

Folder load time: 5.327696 seconds
Idle...Folder load time: 5.424941 seconds

Then quit nemo and reopened.

steve@steve-Inspiron-5580:~$ NEMO_BENCHMARK_LOADING=1 nemo /home/steve/Nemo_Test/1000_pics

** (nemo:5955): WARNING **: 10:24:25.737: Current gtk theme is not known to have nemo support (Cinnamox-Rhino) - checking...
Initializing nemo-image-converter extension
Nemo startup time: 6.736869 seconds
Folder load time: 5.945852 seconds
Idle...Folder load time: 6.042008 seconds

ps aux output for nemo displaying the thumbnailed folder from a fresh start

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve     5955 42.7  5.2 1275420 410260 pts/0  Sl+  10:24   0:44 nemo /home/steve/Nemo_Test/1000_pics

I then had a quick look at the thumbnail cache and notes that the time difference between the first and last thumbnail produced was 1m 34 seconds.

Test 2 - Thumbnail generation on a folder of 5000 images

Directory info

steve@steve-Inspiron-5580:~$ gio info /home/steve/Nemo_Test/5000_pics | grep metadata
  metadata::nemo-default-view: OAFIID:Nemo_File_Manager_Icon_View
  metadata::nemo-icon-view-zoom-level: 6
steve@steve-Inspiron-5580:~$ du --si ~/Nemo_Test/5000_pics
15G	/home/steve/Nemo_Test/5000_pics

Then quit nemo, cleared the thumbnail cache and restarted nemo directly on the target folder.

steve@steve-Inspiron-5580:~$ NEMO_BENCHMARK_LOADING=1 nemo /home/steve/Nemo_Test/5000_pics

** (nemo:22298): WARNING **: 10:37:50.665: Current gtk theme is not known to have nemo support (Cinnamox-Rhino) - checking...
Initializing nemo-image-converter extension
Nemo startup time: 2.115366 seconds
Folder load time: 1.362390 seconds
Idle...Folder load time: 1.621272 seconds

** (gdk-pixbuf-thumbnailer:14550): WARNING **: 10:40:50.355: Could not thumbnail 'file:///home/steve/Nemo_Test/5000_pics/Family_080223_230.JPG': Error interpreting JPEG image file (Not a JPEG file: starts with 0x00 0x00)

(nemo:22298): CinnamonDesktop-WARNING **: 10:40:50.357: Error creating thumbnail for file:///home/steve/Nemo_Test/5000_pics/Family_080223_230.JPG: Error interpreting JPEG image file (Not a JPEG file: starts with 0x00 0x00)

** (gdk-pixbuf-thumbnailer:14553): WARNING **: 10:40:50.428: Could not thumbnail 'file:///home/steve/Nemo_Test/5000_pics/Family_080223_231.JPG': Error interpreting JPEG image file (Not a JPEG file: starts with 0x00 0x00)

(nemo:22298): CinnamonDesktop-WARNING **: 10:40:50.428: Error creating thumbnail for file:///home/steve/Nemo_Test/5000_pics/Family_080223_231.JPG: Error interpreting JPEG image file (Not a JPEG file: starts with 0x00 0x00)

ps aux output for nemo having thumbnailed all pics

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    22298 89.6 23.7 2758336 1866576 pts/0 Sl+  10:37  13:09 nemo /home/steve/Nemo_Test/5000_pics

Thumbnailing took about 13 minutes. Nemo became unresponsive towards the end of the process and didn't recover so I was forced to nemo -q

I then reloaded the folder.

steve@steve-Inspiron-5580:~$ NEMO_BENCHMARK_LOADING=1 nemo /home/steve/Nemo_Test/5000_pics

** (nemo:19151): WARNING **: 10:55:26.031: Current gtk theme is not known to have nemo support (Cinnamox-Rhino) - checking...
Initializing nemo-image-converter extension
Nemo startup time: 27.819082 seconds
Folder load time: 26.968501 seconds
Idle...Folder load time: 27.900667 seconds

ps aux output for nemo displaying the thumbnailed folder from a fresh start

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    19151  105 21.2 2615300 1673416 pts/0 Rl+  10:55   1:07 nemo /home/steve/Nemo_Test/5000_pics

There was no lockup this time so I navigated to the parent folder and back again
ps aux output for nemo having navigated back to parent folder

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    19151  103  4.8 1324892 382804 pts/0  Sl+  10:55   3:06 nemo /home/steve/Nemo_Test/5000_pics

Loading of the pics folder was excessive this time and nemo was unresponsive once loading at finished

Folder load time: 167.125083 seconds
Idle...Folder load time: 183.289171 seconds

ps aux output for nemo displaying the thumbnailed folder again.

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    19151 92.7 22.5 2714828 1772496 pts/0 Rl+  10:55   8:57 nemo /home/steve/Nemo_Test/5000_pics

I then quit nemo with nemo-q and then reopened to check the thumbnail cache. The time difference between the oldest and newest thumbnail was 10m 34 seconds.

Test 3 - The effect of toggling thumbnail visibility on an already thumbnailed folder.

I did this test on the 1000 pic directory given the lock-ups experienced just thumbnailing the 500 pic directory. First I opened the directory in nemo and allowed it to complete thumbnailing before quiting nemo. Then opened the folder from a fresh start.

steve@steve-Inspiron-5580:~$ NEMO_BENCHMARK_LOADING=1 nemo /home/steve/Nemo_Test/1000_pics

** (nemo:29637): WARNING **: 11:16:41.953: Current gtk theme is not known to have nemo support (Cinnamox-Rhino) - checking...
Initializing nemo-image-converter extension
Nemo startup time: 6.781645 seconds
Folder load time: 6.008563 seconds
Idle...Folder load time: 6.013613 seconds

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    29637  107  5.2 1355564 412516 pts/0  Rl+  11:16   0:44 nemo /home/steve/Nemo_Test/1000_pics

I then toggled thumbnail visibility to off

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    29637 60.9  2.9 1159900 230652 pts/0  Sl+  11:16   0:46 nemo /home/steve/Nemo_Test/1000_pics

Then back on. There was quite a lengthy delay before all thumbnails were reloaded.

teve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    29637 61.4  7.6 1603940 602520 pts/0  Sl+  11:16   1:47 nemo /home/steve/Nemo_Test/1000_pics

Noting the increased RAM usage I then toggled off again

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    29637 49.1  3.0 1173596 242400 pts/0  Sl+  11:16   1:53 nemo /home/steve/Nemo_Test/1000_pics

And back on

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    29637 56.7  7.2 1561980 573360 pts/0  Rl+  11:16   3:05 nemo /home/steve/Nemo_Test/1000_pics

Hopefully this is helpful info in identifying bottlenecks. I was disappointed to find that 5000 pics was enough to lock up nemo entirely as I don't think that is a particularly extreme number of pictures to have in a folder.

I'll keep the the two test folders as is so if there in any further testing that would be helpful let me know.

Related open issues - #2245, #1907

[mtwebster]

I've been messing with this on and off for a while (and have quite a few heavily populated pic folders polluting my disk space) - one optimization I'm looking at is how the icon view layout and spacing changes as thumbnails are generated.

Currently if you have a row of short images, once they get thumbnailed (and their placeholder image is replaced), their position relative to the row above them can be adjusted to keep spacing to a minimum. This can happen a great deal in a large folder, and even more depending on the zoom level and width of the window.

I've made a branch recently: #2289 - it uses a strictly fixed spacing between icons - meaning, there is no optimization for image height - spacing is based on the current zoom level's icon size. This seemed to have a noticeable effect on loading time (in a folder that hasn't been previously thumbnailed), as well as an observable reduction in cpu time.

Some data: https://www.dropbox.com/sh/3xsn90kq4unr3u3/AADB4QWUvMxU9EeNAu7ZvSEka?dl=0 - I'm no expert on this type of analysis, but it has become a useful tool. The graphs are slightly interactive if you load them directly in a browser.

When testing this stuff, I also clear the disk cache to take it out of the equation:

sync && echo 3 | sudo tee /proc/sys/vm/drop_caches

I'd be curious about your observation with the pull request - it's not a finished thing, just using it to work out ideas on how to improve this situation. I'll make deb package if you want.

I did actually go back to working on this after your last issue during the beta, though this can be a big time sink with little reward and there were other priorities. At the time I wasn't too surprised at some of the outcomes you had - with only 2gb ram and little swap space, it was inevitable you were going to run out of memory.

[smurphos]

Thanks @mtwebster - I guessed it would probably still be on your radar but thought some actual data would still be helpful. I'll take a look at building from the test branch and seeing how it helps.

I thought I'd have a look at how pix deals with the 5000 pic folder. Pretty well as it turns out. I emptied the thumbnail cache before firing up pix. It thumbnailed the whole folder within a couple of minutes, remained responsive throughout, and does a good job of prioritising thumbnails for files currently in view very quickly. Still demonstrates quite high memory usage once all thumbnails are loaded, but with the memory available on this hardware this doesn't seem to impact it's performance. Not sure if there is anything it the way that pix does things that could be ported into nemo?

steve@steve-Inspiron-5580:~$ ps aux | grep pix |grep -v "grep"
steve     7896 30.2 23.7 2464708 1867688 ?     Sl   13:50   1:39 pix

[mtwebster]

If you want something else to try :)

#2292

This uses a more modern thumbnailing code from gnome's 'gnome-desktop' library. I've been considering backporting it. It seems to take a bit longer to thumbnail everything, but possibly uses less cpu doing so. I have barely tried it yet, just sharing if you're interested.

[smurphos]

I'll do some testing of both during the course of this week.

Sorry I mean't to ask before is there any debugging commands that might help work out what nemo is doing when it is unresponsive on displaying the 5000 pic folder of already thumbnailed images? In my test above although nemo's memory use was high at that point the system wasn't using swap. Eliminating that unresponsiveness I think would go a long way to improving the end user experience.

[smurphos]

Hi - observations with the test-icon-space branch and the 5000 image folder.

1. Having cleared the thumbnail cache, quit nemo and opened the folder loading time was fractionally quicker.

Initializing nemo-image-converter extension
Nemo startup time: 2.066561 seconds
Folder load time: 1.317603 seconds
Idle...Folder load time: 1.590643 seconds

2. Memory use having thumbnailed the folder was reduced by a fair margins. It got there a lot quicker as-well

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    15684 68.1 21.8 2611012 1723880 pts/0 Sl+  06:51   9:15 nemo /home/steve/Nemo_Test/5000_pics

3. Nemo remained responsive throughout - ;-)

4. Memory use navigating to the parent folder

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    15684 55.9  5.4 1314876 428860 pts/0  Sl+  06:51   9:29 nemo /home/steve/Nemo_Test/5000_pics

5. And time to load and memory use navigating back to the 5000 pic folder. The interface did lag briefly a few times during loading but was responsive once done.

Folder load time: 104.946883 seconds
Idle...Folder load time: 109.503717 seconds

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    15684 59.6 26.6 3102556 2097140 pts/0 Rl+  06:51  12:51 nemo /home/steve/Nemo_Test/5000_pics

6. Load time and memory use of loading the folder having quit nemo. Again there was a little bit of lag initially but nemo then remained responsive

Nemo startup time: 76.204590 seconds
Folder load time: 75.320999 seconds
Idle...Folder load time: 77.782844 seconds

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    15019  104 21.3 2620824 1680812 pts/0 Rl+  07:14   1:50 nemo /home/steve/Nemo_Test/5000_pics

7. Thumbnail time stamps - oldest to newest was 10 mins 25 seconds.

Overall this is a good improvement - if nothing else because nemo didn't lock up indefinitely at any point and everything is a bit faster and used a little less RAM. The aesthetics aren't so great particularly for 16:10 images in portrait, but personally that wouldn't bother me.

[smurphos]

Observations with the gnome-thumbnail-lib branch and the 5000 image folder.

1. Initial loading time with the cleared thumbnail cache

Nemo startup time: 1.954372 seconds
Folder load time: 1.221655 seconds
Idle...Folder load time: 1.561405 seconds

2. Memory after thumbnailing. Seemed pretty quick.

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve     9044 82.8 25.7 3055900 2027184 pts/0 Sl+  09:16  10:50 nemo /home/steve/Nemo_Test/5000_pics

3. Nemo is still responsive

4. Memory on navigating to parent folder

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve     9044 62.5  6.5 1458812 512408 pts/0  Sl+  09:16  10:55 nemo /home/steve/Nemo_Test/5000_pics

5. Time to load and memory on going back to the picture folder. Pretty long again with the icon spacing calculations back in the mix. Nemo was unresponsive for about 30 seconds after loading but did come back to life

Folder load time: 141.949962 seconds
Idle...Folder load time: 154.800723 seconds

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve     9044 68.8 26.3 3116180 2075648 pts/0 Rl+  09:16  16:52 nemo /home/steve/Nemo_Test/5000_pics

6. Load time and memory use of loading the folder having quit nemo. Again there was a little bit of lag initially but nemo then remained responsive

Nemo startup time: 94.764691 seconds
Folder load time: 94.003407 seconds
Idle...Folder load time: 100.891455 seconds

steve@steve-Inspiron-5580:~$ ps aux | grep nemo |grep -Ev "grep|desktop"
steve    17536  105 23.2 2774492 1834188 pts/0 Rl+  09:46   2:16 nemo /home/steve/Nemo_Test/5000_pic

7. Thumbmail time stamps - oldest to newest was 12 minutes 2 seconds.

Hello,

I am using nemo-4.4.2-1.fc30.x86_64 (Fedora 30 spin). I have similar problems as described above, but with considerably less number of images in a folder. For example, a folder with 600 images makes nemo completely freeze during thumbnail generation.

I noticed the following things:

1. the nemo thumbnail generation procedure is not multi-threaded, which is a major concern these days with CPU's with lots of cores being quite common. I would guess this is a bottleneck and maybe what is causing nemo to be unresponsive during thumbnail generation.

2. while my images are stored on an XFS raid-1 mirror over SSD drives, the thumbnail procedure appears quite slow. So I booted with Fedora 31 and the default gnome3 desktop environment, the same XFS raid-1 mirror runs considerably faster when gnome3 generates thumbnails... maybe by a magnitude of 100x.

3. my images are quite large, 50 to 100 MB each, so I've set the thumbnail view limit to 100MB, otherwise I couldn't see any thumbnails.

Ideally, it would be nice if nemo could see how many cpu cores are available and fork several thumbnail generators, independent of the main nemo process showing the folder GUI. Care should be taken not to fork too many times, I've seen people set a limit, like use half the cpu cores when there are more than two, but use only one when less than two, etc.

Since nemo kept freezing, I installed nautilus, which is more efficient and has no problems generating thumbnails.

Nemo startup time: 4.197017 seconds
Folder load time: 3.715763 seconds
Idle...Folder load time: 3.739099 seconds

$ ps aux | grep nemo | grep -Ev "grep|desktop"
user      2258  104  1.8 1433272 308948 pts/2  Rl+  23:49   1:19 nemo /home/user/Pictures

[nick-s-b]

Now that nemo doesn't crash on my machine anymore (last of the crash bugs has been fixed many months ago), performance when viewing large folders is the last big nemo issue for me.

I have a ~/Downloads folder that has 2,000+ items in it. I like to view it as thumbnails, reverse sorted by date (latest at top). nemo is basically unusable with it. I have to resort to using Nautilus with any large folder.

Biggest annoyances:

1. When entering a large folder, nemo keeps on sorting files for a very long time. Files are constantly popping up and disappearing as nemo is trying to arrange them. It can take minutes for files to stop rearranging themselves. This does not happen with Nautilus. With other FMs, they display all the files in the correct order nearly instantly and then take their time to generate thumbnails. With nemo, it's constantly re-arranging files and generating thumbnails. You cannot even select files in nemo or double-click on them when this is happening. In nautilus you can click on them and open them just fine.

2. nemo almost always "freezes" on such folders after few minutes. You cannot scroll through them, you cannot select files. If you use your scroll wheel, files will be scrolled maybe 1-2 minutes after. Lag is truly incredible. It's completely unusable for some reason. nemo process is not stopped. It's not consuming 100% of the CPU or anything like that. It's just extremely lagged. The only 'fix' is to use kill on the process.

I'm not sure what the issue is with nemo and large folders but it's nearly impossible to use it with lots of files.

[mtwebster]

I'm working on some improvements that should hopefully make it into the next release. The performance bump in most instances is pretty significant.

it's here if you wanted to try it: #2341 (It's by no means complete or perfect yet but it's reasonably useable).

[mtwebster]

Hi, I just pushed the performance improvements, the relevant commits are:
323d0f0
bb049a4

[mindinsomnia]

Hi, I just pushed the performance improvements, the relevant commits are:
323d0f0
bb049a4

In the words of my people: "You bloody ripper!"

That's amazing, simple fantastic! I've tried and failed to browse large folders in nemo many times and watched it struggling to load and thought "If it just had a fixed vertical spacing layout and only loaded the thumbnails in view, it would load so much faster".

Dude, mtwebster, you're my hero today, thankyou, brb need to go find the LM donate page..

[nick-s-b]

@mtwebster this is amazing! I just compiled it and it and installed it and it's noticeably faster. Thank you!