Broxygen docs

(Moving this to bro-dev).

I've only taken a quick look at your recent changes so far, but that
looks good!

Quick comments:

    - I think we should leave the "other components" out and document
      them separately on the web page. That stuff is pulled out of the
      other git repositories, and doing that manually (as I think one
      would have to, right?) is quite cumbersome and error prone.

    - The changes break the current web pages unfortunately, see:

        http://www.bro-ids.org/documentation-beta/index.html
        http://www.bro-ids.org/documentation-git/index.html

      We should be able to hard-code the beta page to the last commit
      before the merge, and can then keep working on the git version.

    - I suggest to use the non-serif font for the section headlines,
      and using serif only for the page title. That's how the web
      pages are doing as well and looks a bit better IMO.

More later. I'll also start to adding some first documentation to
event.bif to see how that gies.

Robin

   - I think we should leave the "other components" out and document
     them separately on the web page.

By letting Broxygen handle rendering of "other components", it seems to me that it makes it easier from a "documentation bundling" perspective in that you'll always get the component documentation that's associated with a given bro repo commit.

That stuff is pulled out of the
     other git repositories, and doing that manually (as I think one
     would have to, right?) is quite cumbersome and error prone.

I think I'm not foreseeing the same process that you are. If 'www' delegates the rendering of "other components" to the 'bro' repo and that gets linked on the Broxygen main index, then all 'www' has to do is grab the 'html' output as a whole. Currently, there's nothing manual about getting Broxygen to render the correct "other components" because they're just symlinks into git submodules.

   - The changes break the current web pages unfortunately, see:

       http://www.bro-ids.org/documentation-beta/index.html
       http://www.bro-ids.org/documentation-git/index.html

     We should be able to hard-code the beta page to the last commit
     before the merge, and can then keep working on the git version.

Yeah, I was kind of hoping for the Broxygen docs to completely replace the current documentation*/ directories. I mean you'd still have different versions of documentation but each one would be obtained my doing `make broxygen` for a particular commit in a 'bro' repo.

Depending on your response to above issues, want me to take a shot ('www' intimidates me sometimes) at inserting the Broxygen docs?

   - I suggest to use the non-serif font for the section headlines,
     and using serif only for the page title. That's how the web
     pages are doing as well and looks a bit better IMO.

Ok. I think I saw a couple other style differences I can try to hunt down.

+Jon

I think I'm not foreseeing the same process that you are.

I may have just misunderstood how things are done currently. I didn't
really look at it yet but thought I saw somewhere a comment about
having to copy files over manually. Discard my concern if that's not
the case.

has to do is grab the 'html' output as a whole. Currently, there's
nothing manual about getting Broxygen to render the correct "other
components" because they're just symlinks into git submodules.

Even with this model (which is generally fine with me) we'll have one
issue: the submodule docs will reflect whatever the specific revision
is that the Bro supermodule pulls in. That's kind of right from the
supermodule's perspective as that's what that version of the
supermodule is using. But on www, we also list the modules separately
and may want to document different versions there. Say we do a Bro
release. The Bro docs are then frozen at that point and will show up
on www/documentation. However, if we then did an intermediary Broccoli
release, the Bro docs would keep showing the old version. Is there a
way to avoid that when having the submodule docs in Sphinx as well?

Yeah, I was kind of hoping for the Broxygen docs to completely replace
the current documentation*/ directories. I mean you'd still have
different versions of documentation but each one would be obtained my
doing `make broxygen` for a particular commit in a 'bro' repo.

Yes, that's my vision as well (or at least "replace *most* of the
current doc directory" depending on how we decide to handle the
submodules, per above).

Depending on your response to above issues, want me to take a shot
('www' intimidates me sometimes) at inserting the Broxygen docs?

That would great, yes.

(www is starting to intimidate me as well as it's getting quite
complex; moving more into Sphinx should help with that as well.
Generally, if you find opportunity to clean things up or make simpler,
feel free to go ahead. My main objective with www is just to keep
things as automated as possible so that when we're doing updates to
any of the repos, one doesn't always need to remember to go to www and
do certain things there as well. That's where some of the complexity
is coming from.).

Ok. I think I saw a couple other style differences I can try to hunt down.

Thanks (but leaving things like this at lower priority is fine).

Robin

I may have just misunderstood how things are done currently. I didn't
really look at it yet but thought I saw somewhere a comment about
having to copy files over manually. Discard my concern if that's not
the case.

Ah, yeah. In a README I think I said you could copy files into the sphinx source tree to get it to render them since that's technically a way to do it. But the other way I said was just to symlink it and that's the way I currently did it.

Even with this model (which is generally fine with me) we'll have one
issue: the submodule docs will reflect whatever the specific revision
is that the Bro supermodule pulls in. That's kind of right from the
supermodule's perspective as that's what that version of the
supermodule is using. But on www, we also list the modules separately
and may want to document different versions there. Say we do a Bro
release. The Bro docs are then frozen at that point and will show up
on www/documentation. However, if we then did an intermediary Broccoli
release, the Bro docs would keep showing the old version. Is there a
way to avoid that when having the submodule docs in Sphinx as well?

Ok, good point. I think that's going to have to be out of the scope of Sphinx/Broxygen. So we'll have two different ways of rendering "other components":

1) 'www' pulls in certain release revisions of each submodule to render on it's own page
2) 'bro' manages the rendering of submodule READMEs depending on whatever the current submodule pointers are, and then 'www' pulls this in as part of a versioned Bro documentation bundle

+Jon

Yeah, that makes sense. We'll need to keep in mind to not use Sphinx
features in the READMEs so that they still render standalone.

Robin