Precaching folders - fatal error

flyingfishfinger

Hi,
I'm running the plugin that was suggested in another thread to pre-generate the image cache for slow servers. That works fine, although I'm running into a problem with running it for the folder structure. When I do that, it goes fine for a couple hundred folders, then crashes with the following error (always at the same place it seems):

Fatal error: Uncaught ValueError: Epoch doesn't fit in a PHP integer in /jffs/www/index.php:953 Stack trace: #0 /jffs/www/index.php(953): DateTime->getTimestamp() #1 /jffs/www/index.php(967): exif_timestamp('0000:00:00 00:0...') #2 /jffs/www/index.php(1135): get_exif('/tmp/mnt/sda3/P...') #3 /jffs/www/index.php(866): get_files_data('/tmp/mnt/sda3/P...', false, 1029329492, 774, 774, 'f7451495.jpg') #4 /tmp/mnt/sda3/_files/plugins/files.tasks.php(171): get_dir('/tmp/mnt/sda3/P...', true) #5 /tmp/mnt/sda3/_files/plugins/files.tasks.php(301): create_cache('/tmp/mnt/sda3/P...', 2) #6 /tmp/mnt/sda3/_files/plugins/files.tasks.php(301): create_cache('/tmp/mnt/sda3/P...', 1) #7 /tmp/mnt/sda3/_files/plugins/files.tasks.php(310): create_cache('/tmp/mnt/sda3/P...') #8 /jffs/www/index.php(1326): include('/tmp/mnt/sda3/_...') #9 /jffs/www/index.php(1790): get_include('plugins/files.t...') #10 {main} thrown in /jffs/www/index.php on line 953

Since the filenames in the error aren't expanded completely, I can't tell what folder it's failing on. Based on the above, it looks like a folder has an all-zero (illegal?) timestamp, but I don't know how to find it.

Actually, there is a filename in the above. But its creation or modified date as well as that of the folder it is in, are NOT 00:00:00:00..... Is an exif_timestamp something else? Further, if I remove the offending item, another one pops up in its place with the same error.

Any suggestions?
Thanks,
R

Karl

flyingfishfinger I haven't heard of this issue before. I did some searches for the error, and it could be this is related to the platform and/or PHP install and version. Otherwise, could you please send a copy of one if the images to see if I can reproduce it? karl @ photo.gallery

flyingfishfinger Further, if I remove the offending item, another one pops up in its place with the same error.

It's not clear, it already manages to process a few hundred folders, but stops in a specific folder? It would seem like there are some issues in images specifically for this folder, and possibly others, but certainly not all images.

I am doing surgery today, and might not be available to answer for a few days.

flyingfishfinger

Yeah, I also found references to the newer php versions with this error - maybe we can look at that later.

I will try removing the entire folder for a test run and see if it continues. On that note - if I'm caching folders, why does the software need to process or check each image at all?

Good luck on the surgery. No rush here, I have 150k images to preprocess the caching for so that will take some time anyway on my slow router.

Btw, are you German by any chance?

Karl

flyingfishfinger On that note - if I'm caching folders, why does the software need to process or check each image at all?

I don't understand the question. Folders are cached as JSON, images in folders are resized and cached. Two separate caching processes, that can be targeted separately from the tasks plugin. It won't check images if you are not running the task that checks images ... Even if you are, it will only process images that are not yet resized.

flyingfishfinger Btw, are you German by any chance?

Norwegen!

flyingfishfinger

Right, so that's confusing. The error above occurs when I ask to precache FOLDERS. Why is it throwing an error on a JPEG image file then? Precaching images has not (yet) thrown an error.

Karl

flyingfishfinger Right, so that's confusing. The error above occurs when I ask to precache FOLDERS. Why is it throwing an error on a JPEG image file then?

That would be correct. When processing FOLDERS, Files Gallery needs to read all the data from files and images inside the folder, which in turn is displayed in the interface. This includes image meta data, EXIF and dates, and it's somewhere here where the error happens.

When resizing images, it's only running a resize function on the image, nothing more. This requires more processing, but it does not cause any errors in your case.

flyingfishfinger

I removed the entire folder with the errors, and the folder precaching completed without issues! So that's good.

Problem: The folder in question contains about 5k images and is 7GB in size - any suggestions on how to check for what's wrong?

Karl

flyingfishfinger Problem: The folder in question contains about 5k images and is 7GB in size - any suggestions on how to check for what's wrong?

Did it fail with any one of the images? You need to start adding them back slowly, and find one image that causes the error, then send it to me. It could be something in the tasks plugin only.

flyingfishfinger

Interesting behaviour. I started copying the images back into the folder bit by bit.

Eventually it gave the error, so I started saving the "suspected images". At some point, it threw up images that were already processed fine before. To double-check this, I emptied the folder again and added only the "suspect images" back in, and then it ran fine.

Then, I added some files until it broke. So I removed the ones that were added up to the the error, leaving only the ones it already processed. Guess what, it broke again.

Is it possible that the script is modifying something about the images while counting them?

Karl

flyingfishfinger It can't write into the images no. But it will read images for folder cache JSON, and also when resizing images. I have no idea at what point you are at in this process, and I know nothing about the error. How do we know it's not memory related?

flyingfishfinger

I'm just trying to cache this folder. It's a separate question from the memory limit, they result in two totally different outcomes. This one here doesn't crash the server or use any memory, really.

Karl

flyingfishfinger Sorry, I'm currently in hospital, so answers are infrequent and short.

From your messages above, it seems you aren't able to reproduce the issue on one specific image? Instead, it happens randomly when there are a certain amount of images? This to me makes it sound a bit random. I mean, random errors isn't really possible, so I can't make much from this.

Of course, if you can isolate one or more images that causes the error, it would be easy to explain and diagnose.