New iteration of the ActivityPub delivery implementation and interface.
Advantages over previous interface:
* When sending a ByteString body, the sender is explicitly passed as a
parameter instead of JSON-parsing it out of the ByteString
* Clear 3 operations provided: Send, Resend and Forward
* Support for per-actor keys
* Actor-type-specific functions (e.g. deliverRemoteDB_D) removed
* Only the most high-level API is exposed to Activity handler code, making
handler code more concise and clear
Also added in this patch:
* Foundation for per-actor key support
* 1 key per actor allowed in DB
* Disabled C2S and S2S handlers now un-exported for clarity
* Audience and capability parsing automatically done for all C2S handlers
* Audience and activity composition automatically done for Vervis.Client
builder functions
Caveats:
* Actor documents still don't link to their per-actor keys; that should be the
last piece to complete per-actor key support
* No moderation and anti-spam tools yet
* Delivery API doesn't yet have good integration of persistence layer, e.g.
activity is separately encoded into bytestring for DB and for HTTP; this will
be improved in the next iteration
* Periodic delivery now done in 3 separate steps, running sequentially; it
simplifies the code, but may be changed for efficiency/robustness in the next
iterations
* Periodic delivery collects per-actor keys in a
1-DB-transaction-for-each-delivery fashion, rather than grabbing them in the
big Esqueleto query (or keeping the signed output in the DB; this isn't done
currently to allow for smooth actor key renewal)
* No support yet in the API for delivery where the actor key has already been
fetched, rather than doing a DB transaction to grab it; such support would be
just an optimization, so it's low-priority, but will be added in later
iterations
Both Git and Darcs are supported
- Darcs implementation applies right on the bare repo, I haven't tested to make
sure it works right (federated MR demo is going to be only for Git)
- Git implementation clones to temporary repo, runs `git am` on it to apply,
then pushes to the real bare repo (because `git am` doesn't work on bare
repos; I haven't tested yet to see how it handles conflicts; cloning and
pushing should be efficient since the refs are just hardlinked rather than
copied)
For now it's implemented only for Git:
If tracker is a local loom, and a (local or remote) origin repo is specified,
but no patches are provided, then generate them ourselves!
* Clone the (local) target repo
* Add the (local or remote) origin repo as a git remote
* Make sure target branch is an ancestor of the origin branch
* Generate patches for the commits that origin adds on top of target
* Insert them into our DB
What's missing:
- Match patch VCS, origin repo VCS and target repo VCS
- Hook into postPersonInboxR
- If only origin is provided, generate patches / otherwise somehow remember the
commits proposed
Person inbox handler:
- Invite: Parse and insert to inbox
- Grant: Parse and insert to inbox
Repo/Deck/Loom inbox handler:
- Invite: Parse and remember as Collab record in DB for later
- Accept: Send a Grant (and remember it in DB)
Along with inviteC and acceptC, the Invite->Accept->Grant flow is now fully
federated, yay!
What's missing is UI for actually using it. Coming soon.
Giving access now starts with an Invite activity, followed by Accept from the
Invite's recipient. Finally, the resource sends a Grant, which is the actual
OCap.
Only Vervis.Hook (for the hook programs) and Vervis.Application (for the web
app program itself) are exposed now. I suppose more will be needed for tests.
But now there's a clear distinction what's used by other components and what
isn't.
The steps are:
- Parse activity ID and match with the authenticated sender
- For local activity (we got via forwarding), find in DB
- For remote activity, cache in DB
- Insert activity to recipient's inbox
What's not there yet is the actual logic of handling specific activities.
This is such a huge patch, it's probably impossible to tell what it does by
looking at the code. One thing is clear: It changes *everything* :P so here's
an overview:
- There are now 5 types of actors, each having its own top-level route
- So projects, repos, etc. are no longer "under" sharers
- Actor routes are now based on their KeyHashid, there are no "idents" anymore,
i.e. URLs look random and don't contain user or repo names
- No sharers anymore; people and groups are distinct entities not sharing a
common namespace or anything like that
- Project has been renamed to Deck and it simply means a ticket tracker; repos
are no longer "under" projects
- In addition to Person, Group, Repo and Deck, there's a new actor type Loom,
which is a patch tracker; i.e. Repo actors don't manage MRs anymore
- All C2S and S2S is temporarily disabled, because huge changes to the whole
code are required and I'll do them gradually in the next patches
- Since form-based actions are implemented using C2S, they're disabled as well,
so Vervis is now essentially read-only
- Some views have been temporarily removed, e.g. repo history and commit view
- A huge set of DB migrations has been added to adapt the DB to these changes;
I haven't tested them yet on a read DB so there may be errors there; I'll fix
them in the next patches if I find any (probably going to test on the main
instance where Vervis itself is hosted...)
- Some modules got tech upgrades, e.g. LocalActor became a higher-kinded type
and a similar pattern is probably relevant for several other types
- There's an 'Actor' entity in the DB schema now, and all 5 actor types use it
for common things like inbox and outbox
- Although inbox and outbox are used only by Actor, so essentially could be
removed, I haven't removed them; that's because I wonder if at some point
users can have a tree of inboxes much like in email; I don't have an excuse
for Outbox, but anyway, leaving them as is for now
- Workflows, roles and collaborators are partially removed/unused until I
figure out a sane federated way to provide these features
- Since repo routes don't contain a "sharer" anymore, SSH URIs are now simpler,
they already look like user@host/repo regardless of who "controls" that repo
I realized I never intend to leave patterns (e.g. pattern matching in a 'case'
clause) incomplete, i.e. some cases left missing. When I do that it means I
forgot, and I'd like GHC to highlight it by raising an error instead of just
warning. Vervis has lots of warnings so it's hard to detect among them.
I suppose in other kinds of software people sometimes leave incomplete patterns
intentionally / are okay with a runtime exception being thrown? In a web
application, I definitely want to handle all cases, and be in control of how
errors are handled and displayed in UI.
To be honest, this is a huge patch that changes tons of stuff and probably
should have been broken up into small changes. But I already had the codebase
not building, so... just did all of this at once :P
Basically this patch does the following:
- DB migrations for ticket dependency related tables, e.g. allowing a remote
author and a remote child
- Allowing S2S handlers to provide an async continued processing function,
which is executed and the result then added to the debug page
- Most UI and functionality related to ticket deps is disabled, new
implementation being added gradually via ActivityPub
- Improvements to AP tools, e.g. allow to specify multiple hosts for approved
forwarding when sending out an activity, and allow to specify audience of
software-authored activities using a convenient human-friendly structure
- Implementation of S2S sharerOfferDepF which creates a dependency under a
sharer-hosted ticket/patch and sends back an Accept
Before, Push activities were being ignored by all inboxes. I just forgot to add
code to handle them. Now, person inboxes accept them if they're about a
relevant repo (i.e. a repo of which the user is a remote follower; remote
collaboration would be relevant too, but it's not implemented yet).
In Darcs, any command can have a post hook (and a pre hook), and the hook
command can be set using a command-line option to the darcs command that you
run. So, in the Vervis SSH server, if we add a --posthook option when running
`darcs apply` to apply remotely received patches, we get a chance to process
the patch data much like in the git post-receive hook.
The setup this patch creates is similar to the git one: It writes a
_darcs/prefs/defaults file to all Darcs repos, and that defaults file sets the
posthook line for `darcs apply`. The posthook line simply executes the actual
hook program written in Haskell.
The current hook program is a one-liner that prints a line to stdout, so every
time you `darcs push` you can tell the hook got executed. The next step is to
implement the actual hook logic, by reading patch data from the environment
variable in which Darcs puts it.
This patch contains migrations that require that there are no follow records.
If you have any, the migration will (hopefully) fail and you'll need to
manually delete any follow records you have. In the next patch I'll try to add
automatic following on the pseudo-client side by running both e.g. createNoteC
and followC in the same POST request handler.
Here's how it works:
- When Vervis starts, it writes a config file and it writes post-receive hooks
into all the repos it manages
- When a git push is accepted, git runs the post-receive hook, which is a
trivial shell script that executes the actual Haskell program implementing
the hook logic
- The Haskell hook program generates a Push JSON object and HTTP POSTs it to
Vervis running on localhost
- Vervis currently responds with an error, the next step is to implement the
actual publishing of ForgeFed Push activities
FedURIs, until now, have been requiring HTTPS, and no port number, and DNS
internet domain names. This works just fine on the forge fediverse, but it
makes local dev builds much less useful.
This patch introduces URI types that have a type tag specifying one of 2 modes:
- `Dev`: Works with URIs like `http://localhost:3000/s/fr33`
- `Fed`: Works with URIs like `https://dev.community/s/fr33`
This should allow even to run multiple federating instances for development,
without needing TLS or reverse proxies or editing the hosts files or anything
like that.
highligher2 doesn't have a JSON syntax and the JS lexer seems to be failing,
not sure exactly why yet. To have an alternative, I'm adding a Skylighting
option.
- The data returned from activity authentication has nicer types now, and no
mess of big tuples.
- Activity authentication code has its own module now, Vervis.Federation.Auth.
- The sharer inbox handler can now handle and store activities by a local
project actor, forwarded from a remote actor. This isn't in use right now,
but once projects start publishing Accept activities, or other things, it may
be needed.
