Reverse of 'dirs_exclude'

Limezy

Hi !

First of all, congrats for this awesome software. I was getting very tired of Nextcloud and found your Files Gallery product that could be a very good alternative for it, as a "frontend" for my documents using Syncthing as the "sync backend". I'll very probably soon order a licence.

I have been using the 'dirs_exclude' setting (https://www.files.gallery/docs/config/#dirs_exclude), together with the new beta users feature, to create a kind of sharing space for friends or family.

Would it be possible to have a 'dirs_include' setting that would act reversely : every matching path would be included to that user's space ? That would avoid me using some regex trickery which can become quite messy...

That would be great !

PS: I'm aware that this doesn't fit with Files.gallery's original intent, but a feature with unique links to share would be nice too
PPS: Living in Thailand too 😉

Karl

Limezy Would it be possible to have a 'dirs_include' setting that would act reversely : every matching path would be included to that user's space ? That would avoid me using some regex trickery which can become quite messy...

I think this was discussed earlier in another post, and we agreed it could be useful to have dirs_include and files_include instead of making complex "all dirs except" rules for exclude options. Keep in mind, existing options dirs_exclude and files_exclude will become void when dirs_include or files_include are populated, because it means that all files (except those in include regex) will already be excluded. Makes sense?

I might not be able to look into this for next release, which will be available soon, but I will surely look into it for the following release.

I found a couple of posts with examples on how to only include certain files by using exclude rules:
https://forum.files.gallery/d/334-regex-to-only-show-specific-files
https://forum.files.gallery/d/471-open-a-file-with-a-specific-url-structure/7

Limezy PS: I'm aware that this doesn't fit with Files.gallery's original intent, but a feature with unique links to share would be nice too

All files and folders already have physical links. If you mean the possibility to create virtual links that share certain files and folders to certain users (or public) for a specific time period (or not), then this is off the radar for now. This would require a database for starters, and normally such a feature would only work for an application that fully controls the entire file system (Files Gallery just reads your existing file structure).

Limezy

Hi @Karl many thanks for your reply.

Karl we agreed it could be useful to have dirs_include and files_include instead of making complex "all dirs except" rules for exclude options

I'm glad you came to that conclusion !

Karl Keep in mind, existing options dirs_exclude and files_exclude will become void when dirs_include or files_include are populated, because it means that all files (except those in include regex) will already be excluded. Makes sense?

It makes complete sense. Some other tools have a kind of "mixed" way to deal with these kind of rules. Maybe could be interesting for files.gallery ? See for example how xattr filter rules are managed by Syncthing https://docs.syncthing.net/advanced/folder-xattr-filter.html

If no rules are set, it's allowing all by default (just like files.gallery currently)
If one or more rules are set and checked to true, then all the rest below will be excluded by default (include only behaviour)
You can come back to an exclude only behaviour by setting one or more rules checked to false, and have a last bottom rule * checked to true

It's slightly more complicated to document and understand, but in the end if makes more sense than having two contradictory settings in the config file.

Example of how it could look for files.gallery in the config file :

Include only behaviour

<?php
return array (
  'dirs_filters' => array(
    'Dir1/To/Be/Included' => true,
    'Dir2/To/Be/Included' => true,
    // The rest will be excluded
  );
  // Other settings... 
);

Exclude only behaviour

<?php
return array (
  'dirs_filters' => array(
    'Dir1/To/Be/Excluded' => false,
    'Dir2/To/Be/Excluded' => false,
    '*' => true,
    // The rest will be included
  );
  // Other settings... 
);

Karl All files and folders already have physical links. If you mean the possibility to create virtual links that share certain files and folders to certain users (or public) for a specific time period (or not), then this is off the radar for now. This would require a database for starters, and normally such a feature would only work for an application that fully controls the entire file system (Files Gallery just reads your existing file structure).

Yes, I understand and indeed the fact that files.gallery doesn't write anything to the file system is very structural to your design.

Karl

Limezy It's slightly more complicated to document and understand, but in the end if makes more sense than having two contradictory settings in the config file.

Actually, it's not too complicated and isn't necessarily contradictory. My only point was that as soon as dirs_include gets populated, it will already exclude all dirs except the ones specified. Therefore, it's unlikely anyone will need to use both. However, there are cases when both can be used, for example:

'files_include' => '/\.jpe?g$/i', // only show JPG image files
'files_exclude' => '/\(^\.|hidden)/i', // exclude JPG images that start with .dot or contain "hidden"

The rules will run in order 1.files_include 2.files_exclude, and it's entirely fine to combine both. files_include will narrow down the matches, while files_exclude could potentially narrow down matches further. It just needs to be clear to the user that as soon as files_include is populated, files will be excluded by default, unless they match the include regex. In most cases, the user will only use one or the other.

Options dirs_include dirs_exclude files_include files_include should be able to cover any requirement.

Limezy return array ( 'dirs_filters' => array( 'Dir1/To/Be/Included' => true,

I would really prefer to avoid complex array options, especially when the same can already be achieved, even when there are no common patterns in dirs to include or exclude.

'dirs_include' => '/Dir1\/To\/Be\/Included|Dir2\/To\/Be\/Included/',
'dirs_exclude' => '/Dir1\/To\/Be\/Excluded|Dir2\/To\/Be\/Excluded/',

Now, the above would not be logical of course, and unlikely you would use both. However, there are many example of combining both include and exclude. For example:

'dirs_include' => '/Dir1\/subdir|Dir2\/subdir/', // only include two dirs
'dirs_exclude' => '/\/secret|\/\_|\/username/', // additionally, exclude dirs that contain 'secret' or start with _underscore or contain 'username'

If we really need more advanced per-dir include/exclude, this would perhaps instead be part of future CMS-like capabilities when/if I decide to store optional config options per dir. In that case, each dir would have a config that could contain options like "hidden", possibly on a per-user basis. Just theory for now ...

Thanks for contributing!

Limezy

Thanks for your reply. That's very clear and indeed, a system as per your explanation above will 100% cover my current needs. I hope you'll have time to make it for your upcoming release, if not I'll wait a bit more !

In the remote future case each dir would have its own config file or flag file, then I believe a share link could be done even without write access for Files.gallery. Whenever a .fg_share flag is put in a directory by the user, then Files.gallery would show a sharing link UX which gives a "random" link, that "random" link being the path of the shared dir encrypted using a public key. When the link is accessed from outside, Files.gallery would use the private key to retrieve the relevant path and serve it to the visitor. The flag file could contain an expiry date for share that Files.gallery would check against before serving the path. (I hope my idea is clear).

Anyway, thanks and happy to see new features / updates coming.

Limezy

Hi @Karl I've just read the release post of v12. Thanks for this new version !
It seems that you didn't have time yet to include the above discussed dirs_include and files_include additional config features. Hopefully they will be released with v13 😉

Karl

Limezy It seems that you didn't have time yet to include the above discussed dirs_include and files_include additional config features. Hopefully they will be released with v13 😉

True. I wanted to get this login-specific version out without distractions first. It looks like there will be 0.12.1 and I could perhaps sneak this feature in there ...

Limezy

Sure thing, no pressure and thanks for considering the feature !

Karl

Limezy Released new version 0.12.1 with new config options files_include and dirs_include.

Limezy

Legend ! I'll try that tonight 😉 Thank you so much

Limezy

I have troubles including a folder that has spaces in the name.
Is this case confirmed to work on your side ?

Karl

Limezy I have troubles including a folder that has spaces in the name.
Is this case confirmed to work on your side ?

What kinda trouble? You mean you are trying to include a folder with spaces in the name, but it's still excluded?

I tried from here, both the below worked:

'dirs_include' => '/some dir/', // name with empty space
'dirs_include' => '/some\sdir/', //  \s also represents empty space in regex

Limezy

Thanks for your reply.
I tried both with space and \s. I must have done something wrong on my side, I'll recheck !

Limezy

Sorry to bother you again. I've tried your examples above, and it works.
However, I can't make the include logic work for subdirectories (whether they are with spaces or not)

'dirs_include' => '/main dir\/some dir/', // name with empty space
'dirs_include' => '/main\sdir\/some\sdir/', //  \s also represents empty space in regex

The docs say :

The regex is tested on the folder path relative to the root.

So, my above examples should work if I want to include /main dir/some dir/, shouldn't they ?

Karl

Limezy The behavior is correct, buy your rule is a paradox. By assigning /main dir\/some dir/ you are telling the application "Only allow access to that single directory, by no means any other dir, or parent dirs of this dir". The dir main dir is not included, so Files Gallery won't create a menu item, which literally blocks the subfolder from displaying in the menu.

So my question, is main dir supposed to be accessible or not? If it's supposed to be included (visible), then of course your rule should instead look like this 'dirs_include' => '/main dir/'. If not, then it won't display in the menu of course. Surely you see the logical problem here.

The following in browser will work ?main%20dir/some%20dir, because it's a direct link to the only dir that is included. This however ?main%20dir will not work, because that dir is not included in your regex. This is correct behavior.

The dir is correctly accessible:

The parent dir has not been included, so you will get an error message:

So in your case, why not use 'dirs_include' => '/main dir/'? How are you expecting this dir to be accessible if it's not included?

I can see that dirs_include can be a bit more challenging than dirs_exclude, but definitely very helpful!

Limezy

Karl The behavior is correct, buy your rule is a paradox.

Of course you are right, now I feel so stupid... I was wrongly assuming that when a dir is included, the navigation path leading to it from the root would be activated, which is not the case.

I got my usecase working by using /^main dir(?:\/some dir)?$/, which matches main dir and main dir/some dir but doesn't match main dir/another dir !

Karl

Limezy I got my usecase working by using /^main dir(?:\/some dir)?$/, which matches main dir and main dir/some dir but doesn't match main dir/another dir !

I think perhaps you are going about this a bit wrong. Keep in mind, this is a regex. If you want to match all subfolders of a specific folder, you don't need to add all the subfolders to the regex.

✅ This would allow all subfolders of main dir, "allow this dir, and any paths that start with this path".

'dirs_include' => '/^main dir(\/|$)/'

🚫 This would allow only the main dir, without subdirs:

'dirs_include' => '/^main dir$/'

If you want to include the main dir and SOME subdirs (but not all), you may need to enter them individually:

'dirs_include' => '/^(main dir$|main dir\/some dir|main dir\/another dir)/'

But now it's getting a bit tedious. Why not then combine dirs_include and dirs_exclude?

'dirs_include' => '/^main dir(\/|$)/', // include "main dir" and all subfolders in this path
'dirs_exclude' => '/^main dir\/(excluded_subdir_1|excluded_subdir_2)/', // ... but exclude these specific dirs ...

PS! What's the difference between the two below?

'dirs_include' => '/^main dir(\/|$)/' // matches the main dir and all sub dirs
'dirs_include' => '/^main dir/'  // matches the main dir and all sub dirs, but could also match "main dirsomething"