Until now I've managed to avoid having to implement this piece quite
easily. The only temptation to implement it has been pseudo-client
features, which are basically just a workaround until Vervis gets a
separate client app.
But now, Person actors need to be able to automatically send
delegator-Grants when they receive a direct-Grant from a resource they
either asked to Join or got an Invite to. So, this creates a need to
track these joins and invites on the server side.
I'm not checking the exact minimal pieces necessary for
delegator-Grants. Instead, I'm starting a full implementation of the
person-side of Collab records (AKA direct-Grants), called Permit
records. This will also allow very convenient display of these Grants
and the extensions received through them, as well as UI buttons such as
accepting/rejecting a received Join or Invite.
Until now, adding a direct collaborator to a Project worked exactly like
with components: Invite or Join, then Accept, finally the direct-Grant.
I missed the fact that much like with project-component relationships,
projects (and teams) need to be able to send extension-Grants to their
direct collaborators.
So in Project's Grant handler it now:
- Recognizes the delegator-Grant coming from a new collaborator
- Sends extension-Grants, using the delegator-Grant as the capability
- When getting a new component and sending extension-Grants for it to
direct collaborators, Project uses their delegator-Grants as
capability
And in Project's Accept handler, it no longer sends extension-Grants
(because it doesn't yet have the collaborator's delegator-Grant at this
point).
NOTE, THIS TEMPORARILY BREAKS grant chains: If you create a Project and
add a Deck to it, you won't get an extension-Grant-for-the-Deck from the
Project, because the Project doesn't yet have your delegator-Grant.
The next commits will implement the Person-side of Collab records, and
will cause Person actors to automatically send the delegator-Grant,
fixing the break.
deckResolve now supports Resolve requests that use extension-Grants. It
iterates the verifies the Grant-chain using the process described in the
ForgeFed specification.
Vervis currently supports only direct grants. The new process supports
delegation chains as well. This commit just implements the new process
as a new function, without yet using it anywhere. The next commits will
plug it into Deck actor handlers.
Ticket closing can now be done via the new C2S, and the "Close ticket"
button on TicketR page is back, and uses that new C2S.
S2S, C2S and pseudo-client are implemented for both Deck and Loom, but
the actual button and POST handler are provided here only for Deck. Will
add ones for Loom soon, as needed.
This is to allow getting rid of the old C2S offerTicketC and write a C2S
Offer handler in the new actor system.
And that is to allow ticket opening and closing to work, so that it can
use delegated OCAPs too, as a first demo of delegated OCAPs in action.
Until now the code GETs the collabs URI to find the resource, but it
didn't make sure the URI was really the collabs URI specified by the
resource. This commit adds the check.
Until now, the resource object itself would be specified. This no longer
works, because it's unclear whether we're adding/removing a collaborator
or a component.
From now on, adding a collaborator is done by pointing to the resource's
'collaborators' URI, not to the resource itself
I was writing a topicLocalInvite handler when I realized how cumbersome
it's becoming, to have separate handlers for local activities. While it
allows me to pick custom specific message names and parameters (which is
why I took that approach in the first place), it causes a lot of
duplication and complexity (because I have to write the remote-activity
handlers anyway; adding local ones doesn't reduce complexity).
So this commit switches the entire system to communicate only using
AP/FF activities, including between local actors.
Limitations:
- Only jcs-eddsa-2022 is supported for now, can add more easily if
needed
- Not verifying forwarded local activities, only remotely-authored ones
- Fetching key and actor with HTTP without using the DB cache, because
the current cache system is deprecated and I haven't yet created the
new one
Limitations:
- Verification of proofs not implemeneted yet, just generation.
Verification probably coming in the next commit.
- Only jcs-eddsa-2022 is supported. Can add more cryptosuites once
they're updated for the requirements of the VC Data Integrity spec.
- Bug: The proofs aren't stored in the DB versions of outgoing activities, i.e.
HTTP GETing an activity won't include the proof. Probably not urgent
to fix. Ideally, change the whole PersistJSONObject/Envelope/etc.
thing to allow to serialize the activity exactly once.
This patch makes Vervis temporarily unusable, because all actors' inbox POST
handlers use the new system, but the actual federation handler code hasn't been
ported. The next patches will port all the S2S activities supported so far, as
well as C2S.
- When pushing to a repo, a Push activity is now automatically published
- The 'actor' is now the repo, and 'attributedTo' specifies the person who
pushed
- No need for 'context' in the Push anymore, since it's always the 'actor'
- 'target' now specifies the branch as a Branch object rather than URI (since
Vervis doesn't keep AS2 objects for branches anymore)
- I deleted 'pushCommitsC' (from Vervis.API) because the code for preparing and
pushing an activity is so simple with the new delivery API, doesn't need a
dedicated pushCommitsC function
- The generated Push activity does generate an HTML summary, unlike all other
generated activities (in which I removed the summary generating code); I'm
still unsure whether to bring back those summaries (extra code to write, for
a problematic feature that may become useless when the new UI comes)
When looking up a specfic actor record for a given ActorId, you're pretty much
guaranteed to find the actor if it exists, because there's 1 function in the
codebase that handles this. Whenever a new actor type is added, which is a rare
event, that function gets updated.
But when mass-selecting actors using Esqueleto? Then, you need to LeftOuterJoin
by yourself on each actor type. This is both ugly and error prone, because all
those places in the codebase need to be updated when adding an actor type. The
only downside is that it means O(n) DB queries instead of O(1).
Perhaps there's some elegant way to "add" the specific-actor Joins to a given
Esqueleto query. Something to do some other time, as an optimization, if the
need arises.
Per-actor keys are now fully supported in Vervis! Caveats:
- The HTTP Signature keys produced by Vervis are Ed25519 keys; software that
expects only RSA keys will fail here
- Like instance keys, per-actor keys are currently served in separate
documents, not embedded in the actor document; so software that expects
embedded keys will fail here
New iteration of the ActivityPub delivery implementation and interface.
Advantages over previous interface:
* When sending a ByteString body, the sender is explicitly passed as a
parameter instead of JSON-parsing it out of the ByteString
* Clear 3 operations provided: Send, Resend and Forward
* Support for per-actor keys
* Actor-type-specific functions (e.g. deliverRemoteDB_D) removed
* Only the most high-level API is exposed to Activity handler code, making
handler code more concise and clear
Also added in this patch:
* Foundation for per-actor key support
* 1 key per actor allowed in DB
* Disabled C2S and S2S handlers now un-exported for clarity
* Audience and capability parsing automatically done for all C2S handlers
* Audience and activity composition automatically done for Vervis.Client
builder functions
Caveats:
* Actor documents still don't link to their per-actor keys; that should be the
last piece to complete per-actor key support
* No moderation and anti-spam tools yet
* Delivery API doesn't yet have good integration of persistence layer, e.g.
activity is separately encoded into bytestring for DB and for HTTP; this will
be improved in the next iteration
* Periodic delivery now done in 3 separate steps, running sequentially; it
simplifies the code, but may be changed for efficiency/robustness in the next
iterations
* Periodic delivery collects per-actor keys in a
1-DB-transaction-for-each-delivery fashion, rather than grabbing them in the
big Esqueleto query (or keeping the signed output in the DB; this isn't done
currently to allow for smooth actor key renewal)
* No support yet in the API for delivery where the actor key has already been
fetched, rather than doing a DB transaction to grab it; such support would be
just an optimization, so it's low-priority, but will be added in later
iterations
It causes first login after verification to redirect to registration page,
which is weird, and the default Yesod falue is False, so, switching back to
False
Repo/Loom/Deck created one in their migrations, but CollabTopicAccept isn't
itself a topic, it's just a helper, so it needs to *find* the relevant
CollabTopicLocal rather than create a new one.
Both Git and Darcs are supported
- Darcs implementation applies right on the bare repo, I haven't tested to make
sure it works right (federated MR demo is going to be only for Git)
- Git implementation clones to temporary repo, runs `git am` on it to apply,
then pushes to the real bare repo (because `git am` doesn't work on bare
repos; I haven't tested yet to see how it handles conflicts; cloning and
pushing should be efficient since the refs are just hardlinked rather than
copied)
For now it's implemented only for Git:
If tracker is a local loom, and a (local or remote) origin repo is specified,
but no patches are provided, then generate them ourselves!
* Clone the (local) target repo
* Add the (local or remote) origin repo as a git remote
* Make sure target branch is an ancestor of the origin branch
* Generate patches for the commits that origin adds on top of target
* Insert them into our DB
In offerTicketC it can be used for fetching commits from the remote origin
repo, by knowing its clone URI
Only HTTP clone URIs are supported for now, because it's enough for finishing
the federated MR implementation. Apparently user@host:path isn't a valid URI
and I'll later add a parser for that
What's missing:
- Match patch VCS, origin repo VCS and target repo VCS
- Hook into postPersonInboxR
- If only origin is provided, generate patches / otherwise somehow remember the
commits proposed
RepoSourceR, for a repo that doesn't have a loom, lists looms that want to
serve that repo with buttons for bidirectionally linking the repo to a loom
Once linked, the repo navbar has a Patches/MRs link pointing to the LoomClothsR
of the linked Loom
Person inbox handler:
- Invite: Parse and insert to inbox
- Grant: Parse and insert to inbox
Repo/Deck/Loom inbox handler:
- Invite: Parse and remember as Collab record in DB for later
- Accept: Send a Grant (and remember it in DB)
Along with inviteC and acceptC, the Invite->Accept->Grant flow is now fully
federated, yay!
What's missing is UI for actually using it. Coming soon.
Giving access now starts with an Invite activity, followed by Accept from the
Invite's recipient. Finally, the resource sends a Grant, which is the actual
OCap.
I was going to link the matching Accept tables to them, but then switched to
the Invite-Accept-Grant model and going to implement it in the next patches. So
I'm committing these new tables just in case I decide to revert to the current
model.
The steps are:
- Parse activity ID and match with the authenticated sender
- For local activity (we got via forwarding), find in DB
- For remote activity, cache in DB
- Insert activity to recipient's inbox
What's not there yet is the actual logic of handling specific activities.
Previously there was just CollabTopicAccept, which worked only for local topics
but pretended to apply to both, due to directly pointing to Collab, thus
possible to insert rows even if there's a CollabTopicRemote.
The new situation is a new CollabTopicLocal table to which the local topic
things point, thus keeping the local and remote data separate and difficult to
confuse.
Since that table being added, a Collab without a CollabTopicAccept is
considered an open proposal/invitation waiting for completion. So inserting the
CollabTopicAccept is now required for making the Grant valid and active.
This is such a huge patch, it's probably impossible to tell what it does by
looking at the code. One thing is clear: It changes *everything* :P so here's
an overview:
- There are now 5 types of actors, each having its own top-level route
- So projects, repos, etc. are no longer "under" sharers
- Actor routes are now based on their KeyHashid, there are no "idents" anymore,
i.e. URLs look random and don't contain user or repo names
- No sharers anymore; people and groups are distinct entities not sharing a
common namespace or anything like that
- Project has been renamed to Deck and it simply means a ticket tracker; repos
are no longer "under" projects
- In addition to Person, Group, Repo and Deck, there's a new actor type Loom,
which is a patch tracker; i.e. Repo actors don't manage MRs anymore
- All C2S and S2S is temporarily disabled, because huge changes to the whole
code are required and I'll do them gradually in the next patches
- Since form-based actions are implemented using C2S, they're disabled as well,
so Vervis is now essentially read-only
- Some views have been temporarily removed, e.g. repo history and commit view
- A huge set of DB migrations has been added to adapt the DB to these changes;
I haven't tested them yet on a read DB so there may be errors there; I'll fix
them in the next patches if I find any (probably going to test on the main
instance where Vervis itself is hosted...)
- Some modules got tech upgrades, e.g. LocalActor became a higher-kinded type
and a similar pattern is probably relevant for several other types
- There's an 'Actor' entity in the DB schema now, and all 5 actor types use it
for common things like inbox and outbox
- Although inbox and outbox are used only by Actor, so essentially could be
removed, I haven't removed them; that's because I wonder if at some point
users can have a tree of inboxes much like in email; I don't have an excuse
for Outbox, but anyway, leaving them as is for now
- Workflows, roles and collaborators are partially removed/unused until I
figure out a sane federated way to provide these features
- Since repo routes don't contain a "sharer" anymore, SSH URIs are now simpler,
they already look like user@host/repo regardless of who "controls" that repo
* Publish a Create activity and respond with a Grant activity
* postProjectsR reuses that code
* No automatic following at the moment
* Workflow and role specified in new project form are ignored for now
* Can't create tracker under a group yet, just under the user
Ugh, that module is such a horrible mess... I hope to turn it soon into
something sane. Is there some generic non-clumsy way restructure the AP
parser/encoder API?
For now, making these ugly changes to support the represenation of
Create {TicketTracker}, which I'm about to implement.
I realized I never intend to leave patterns (e.g. pattern matching in a 'case'
clause) incomplete, i.e. some cases left missing. When I do that it means I
forgot, and I'd like GHC to highlight it by raising an error instead of just
warning. Vervis has lots of warnings so it's hard to detect among them.
I suppose in other kinds of software people sometimes leave incomplete patterns
intentionally / are okay with a runtime exception being thrown? In a web
application, I definitely want to handle all cases, and be in control of how
errors are handled and displayed in UI.
