dotcomUNDERGROUND Issue should not be with media I think. Its simple jpeg and mp4 files which load perfectly when I am browsing through. The 500 issue only gets triggered when I am refreshing a (comparatively) large directory.
Well, if you did what I said, we could resolve it quickly. For example, is FFmpeg (mp4 files) could be causing an error, who knows? You didn't even tell me how many files are in the folder.
There is no such thing as "500 internal server error" for large folders. If you have a massive folder, let's say 10.000 files, maybe it could take several seconds extra for the page to respond. If you had a whopping 1.000.000 files, it would take much longer, and eventually TIMEOUT on 30 seconds (depending on your server settings). There is no such thing as 500 internal server error response for "large folders".
Without a link, I can't do much more. You need to start eliminating potential causes.