Loading directories

For policy files, we now have sub-directory structures like this:

    foo/base.bro
    foo/ext1.bro
    foo/ext2.bro
    foo.bro

foo.bro contains

    @load foo/base.bro
    @load foo/ext1.bro
    @load foo/ext2.bro

And the effect is that "@load foo" pulls in all of the scripts at
once, which is neat.

What I'm wondering hiwer is whether we should move "foo.bro" into the
subdirectory under some magic name (like init.bro or __init__.bro or
__package__.bro or ...). And then allow to write "@load foo" (with
"foo" now being the directory name) with the effect of loading
"foo/init.bro". In addition, inside foo/* we could allow "@load
ext1.bro" instead of "@load foo/ext1.bro".

(Coincidentally, this would be quite like Python does it ... :slight_smile:

Robin

What I'm wondering hiwer is whether we should move "foo.bro" into the
subdirectory under some magic name (like init.bro or __init__.bro or
__package__.bro or ...)

That makes a lot of sense to me. I've been thinking the same thing for a while, those wrapper scripts really litter things up for not much benefit.

In addition, inside foo/* we could allow "@load
ext1.bro" instead of "@load foo/ext1.bro".

Hm, I'm not totally sure I like this. It could get confusing when two scripts have the same name. If you want to load the software framework and you are in http/init.bro how do you do it if there is a file named http/software.bro?

(Coincidentally, this would be quite like Python does it ... :slight_smile:

I can go for copying syntax and idioms to make the language feel a bit more familiar and comfortable. :slight_smile:

  .Seth

Good question. :slight_smile: Hmm... Then how about just using "@load ./ext1.bro"
inside the directory. That should already work and also avoid the need
to repeat the directory name.

Robin

That sounds better to me.

  .Seth

What I'm wondering hiwer is whether we should move "foo.bro" into the
subdirectory under some magic name (like init.bro or __init__.bro or
__package__.bro or ...). And then allow to write "@load foo" (with
"foo" now being the directory name) with the effect of loading
"foo/init.bro".

I think that makes sense to me.

A minor detail would be choosing which gets preference: a file or a directory. E.g. say you had a "foo.bro" file and a "foo/" directory that live on the same path, what does "@load foo" actually load?

In addition, inside foo/* we could allow "@load ext1.bro" instead of "@load foo/ext1.bro".
(Coincidentally, this would be quite like Python does it ... :slight_smile:

Initially only seemed to me like a minor convenience is gained from this, but I can't really argue against making things more analogous to an existing language.

- Jon

What I'm wondering hiwer is whether we should move "foo.bro" into the
subdirectory under some magic name (like init.bro or __init__.bro or
__package__.bro or ...)

How about just foo/foo.bro? That would avoid problem of referring to the
wrong instance of the script due to confusion regarding what directory one
is working out of.

BTW, if foo.bro itself pulls in ext1.bro etc., then what if the user doesn't
*want* the ext1.bro functionality? Or is the model that they have to use
it, and it's just split out into another script for keeping each script
bite-sized?

    Vern

BTW, if foo.bro itself pulls in ext1.bro etc., then what if the user
doesn't *want* the ext1.bro functionality? Or is the model that they have to
use it, and it's just split out into another script for keeping each
script bite-sized?

I remember seeing that the @unload directive exists and looks like it's supposed to prevent future @load's of a given script. Don't know if it works, but that could be a way for a user to blacklist the loading of certain scripts in a package.

- Jon

How about just foo/foo.bro?

Hmm ... I think I'd prefer some visual indicator that the script is
special, i.e., it gets loaded when the directory is specified. Having
a consistent name for that would point that out.

BTW, if foo.bro itself pulls in ext1.bro etc., then what if the user doesn't
*want* the ext1.bro functionality?

The model is that by "@load foo" he gets the full set. If he doesn't
want that, he can still pick his pieces individually ("@load
foo/base.bro; @load foo/ext2.bro").

The @unload may do it too, but I don't like that statement. :slight_smile:

Robin

> BTW, if foo.bro itself pulls in ext1.bro etc., then what if the
> user doesn't *want* the ext1.bro functionality?

The model is that by "@load foo" he gets the full set. If he doesn't
want that, he can still pick his pieces individually ("@load
foo/base.bro; @load foo/ext2.bro").

The @unload may do it too, but I don't like that statement. :slight_smile:

If it's just the name of @unload (I'd agree the sound of it is misleading), it could be offered as a different command that does the same thing.

I just think it's nice that a user can choose which approach is best for them: either a whitelist that picks individual pieces of a "package" or a blacklist that blocks only certain scripts of a "package". The blacklist approach allows a user to automatically start loading new scripts that get added to a package in future releases, but the whitelist must be manually updated in that situation.

- Jon

If it's just the name of @unload (I'd agree the sound of it is
misleading), it could be offered as a different command that does the
same thing.

The name is one piece, but it can also be confusing if one unloads a
script that some other one depends on and then suddendly doesn't work
as expected anymore (or not at all). And the order of @loads and
@unloads matters, which isn't nice either.

I just think it's nice that a user can choose which approach is best
for them: either a whitelist that picks individual pieces of a
"package" or a blacklist that blocks only certain scripts of a
"package".

That's right, I'm not saying nobody should use @unload, but I don't
want it to be the only part of the story. Having both options is
indeed helpful.

Robin

I've just commited this to master. The magic file name is
"__load__.bro".

However, it turns out that "@load ./abc.bro" doesn't work because the
cwd doesn't get adjusted when loading a file. That isn't easy to add,
so I'm leaving that as it is. Seems fine for now.

Robin