forge/Vervis

Fork 0

fr33domlover 5712b06299 FEDERATION:.md: Add to-do item about the ticket followres collection name

2019-05-29 07:50:47 +00:00

20 KiB

Raw Blame History

ForgeFed/ActivityPub Federation in Vervis

At the time of writing, here's the current status of federation implemented in Vervis.

Summary:

To post a comment on a local ticket (same server), log in and browse to the ticket's page and use the ticket reply form, the regular way it's been on Vervis.
To post a comment on a remote ticket (other server), log in and browse to the /publish page, fill the form and the comment will be delivered to the right place

For more details, read below.

Federation triggered by regular UI

Ticket comments are federated. If you submit a ticket comment, local and remote users who previously commented on the same ticket will get your comment delivered to their inboxes. The user who created the ticket's project will have it delivered to them too.
If you comment on a ticket, you automatically become a ticket follower, and all future comments on the ticket will be delivered to your inbox.
You can see users' outboxes.
You can see your inbox.
If you create a project, all comments on all tickets of the project will be delivered to your inbox.
There is UI for notifications about comments on tickets you commented on or whose projects you created. However there's no JS to display them in real-time and no email integration.

The ticket comment UI allows to see tickets and comments, and if you're logged in, you can post new comments. If you wish to post a comment on a ticket hosted on another server, not the one on which your account is hosted, see the dedicated federation pages listed below.

GET endpoints

GET /publish

A page where you can write and publish a ticket comment, either on a local ticket (i.e. a ticket on a project hosted on the same server as your account) or on a remote ticket (i.e. a ticket on a project hosted on some other server).

GET /inbox

A test page that displays received activities and the result of their processing.

GET /s/joe/inbox

A page that displays your personal inbox. It should list all ticket comments on projects you've created and and ticket comments on tickets you previously commented on.

GET /s/joe/outbox

A page that displays your personal outbox. It should list all the activities you're published, all ticket comments you've made.

POST endpoints

POST /s/joe/outbox

Personal endpoint for publishing ticket comments. When you submit the form in the /publish page, this is where it is sent. In the future you'll be able to see the content of your outbox, and other people will be able to see the public items in your outbox.

You can access this endpoint without using the /publish page, but Vervis doesn't have OAuth2 support yet, so you'll need to log in first and grab the cookie, and send it along with the request.

POST /s/joe/inbox

Personal endpoint to which other servers deliver ticket comments for you to see. These are comments on tickets on which you previously commented, and thus automatically became a follower of thosr tickets.

POST /s/joe/p/proj/inbox

Per-project inbox, to which projects receive ticket comments from other servers. If someone on another server publishes a comment on your project, then your project will receive the comment at this endpoint and the comment will be displayed when you visit the ticket page.

Spec

Federation in Vervis is done using ActivityPub. Below comes a description of the details that aren't already common on the Fediverse. The details are written informally in the form of short simple proposals.

(A) Authentication

Vervis uses HTTP Signatures to authenticate messages received in inboxes. The Host, (request-target), Date and Digest headers are required to be present and used in the signature, and the Digest header must be verified by computing the hash of the request body. Other headers may need signing too, as specified in the proposals below.

The publicKeyPem field maps to the PEM encoding of the key. The PEM encoding contains not just the key itself, but also a code specifying the key type. The Fediverse de-facto standard is RSA, more precisely PKCS#1 v1.5, and used with the SHA-256 hash algorithm. This is often referred to as RSA-SHA256.

(1) Actor key(s) in a separate document

Allow an actor's signing key to be a separate document, rather than embedded in the actor document. In Vervis, the use of that is for server-scope keys (see proposal below), but otherwise, an embedded key is just as good.

GET /users/aviva/keys/key1

{ "@context":     "https://w3id.org/security/v1"
, "@id":          "https://example.dev/users/aviva/keys/key1"
, "@type":        "Key"
, "owner":        "https://example.dev/users/aviva"
, "publicKeyPem": "-----BEGIN PUBLIC KEY----- ..."
}

GET /users/aviva

{ "@context":
    [ "https://www.w3.org/ns/activitystreams"
    , "https://w3id.org/security/v1"
    ]
, "id":                "https://example.dev/users/aviva"
, "type":              "Person"
, "preferredUsername": "aviva"
, "name":              "Aviva"
, "inbox":             "https://example.dev/users/aviva/inbox"
, "outbox":            "https://example.dev/users/aviva/outbox"
, "publicKey":         "https://example.dev/users/aviva/keys/key1"
}

Authentication requirements:

The keyId from the signature header matches the @id in the document you receive
The and key and the owner actor IDs are on the same host
They key specifies the owner, and the owner actor's publicKey links back to the key

(2) Multiple actor keys

Allow an actor to specify more than one key, or no key at all. This means that when you examine the owner actor of the key, you verify the actor links back to the key by checking that the key is listed among the actor's keys (instead of requiring/expecting only a single key to be specified by the actor).

The reason this is used in Vervis is for key rotation using a pair of server-cope keys (see proposal below).

When used along with proposal A.1, each key may be either embedded in the document, or a URI specifying the ID of a key defined in a separate document.

Actors that never need to post activities can simply not specify any keys at all.

GET /users/aviva

{ "@context":
    [ "https://www.w3.org/ns/activitystreams"
    , "https://w3id.org/security/v1"
    ]
, "id":                "https://example.dev/users/aviva"
, "type":              "Person"
, "preferredUsername": "aviva"
, "name":              "Aviva"
, "inbox":             "https://example.dev/users/aviva/inbox"
, "outbox":            "https://example.dev/users/aviva/outbox"
, "publicKey":
    [ { "id":           "https://example.dev/users/aviva#main-key"
      , "type":         "Key"
      , "owner":        "https://example.dev/users/aviva"
      , "publicKeyPem": "-----BEGIN PUBLIC KEY----- ..."
      }
    , "https://example.dev/users/aviva/extra-keys/extra-key1"
    , "https://example.dev/users/aviva/extra-keys/extra-key2"
    ]
}

(3) Server-scope actor key

Allows to have actor keys that can be used to sign (and verify) activities of any actor on the server, not limited to any specific actor. That allows to have some small constant number of keys on the server, which is very easy to manage and makes key rotations very cheap. It also saves storage of many local and remote actor keys.

In the common Fediverse situation, there's a separate key for each actor, but all of these actor keys are managed by a single entity, the server. The signatures aren't made on users' devices using private keys they keep to themselves. They're made by the server, using private keys the server generates.

Server-scope keys are made by the server too. The server makes the signatures, using a private key it generates and maintains. The server is the owner of the key, and a part of the signed message is the ID of the actor on whose behalf the message is being sent. Since the actor isn't specified by the key, the actor ID is instead placed in a HTTP header. And the actor still has to list the key under publicKey as usual.

GET /key1

{ "@context":
    [ "https://w3id.org/security/v1"
    , { "isShared": "https://peers.community/as2-ext#isShared"
      }
    ]
, "@id":          "https://example.dev/key1"
, "@type":        "Key"
, "owner":        "https://example.dev"
, "isShared":     true
, "publicKeyPem": "-----BEGIN PUBLIC KEY----- ..."
}

GET /users/aviva

{ "@context":
    [ "https://www.w3.org/ns/activitystreams"
    , "https://w3id.org/security/v1"
    ]
, "id":                "https://example.dev/users/aviva"
, "type":              "Person"
, "preferredUsername": "aviva"
, "name":              "Aviva"
, "inbox":             "https://example.dev/users/aviva/inbox"
, "outbox":            "https://example.dev/users/aviva/outbox"
, "publicKey":
    [ { "id":           "https://example.dev/users/aviva#main-key"
      , "type":         "Key"
      , "owner":        "https://example.dev/users/aviva"
      , "publicKeyPem": "-----BEGIN PUBLIC KEY----- ..."
      }
    , "https://example.dev/users/aviva/extra-keys/extra-key1"
    , "https://example.dev/users/aviva/extra-keys/extra-key2"
    , "https://example.dev/key1"
    ]
}

Requirements for a server-scope key:

Its owner is the top-level URI of the server, of the form https://HOST
The isShared property is true
The key is in its own document, not embedded in an actor

Requirements for authentication using a server-scope key:

The actor ID is specified in the ActivityPub-Actor HTTP header
The actor and key are on the same server
That header is included in the HTTP Signature in the Signature header
That actor lists the key (as one of the keys) under publicKey
In the payload, i.e. the activity in the request body, the activity's actor is the same one specified in the ActivityPub-Actor (unless the activity is forwarded, see proposal B.2 about inbox forwarding)

(4) Actor key expiration and revocation

Allow to improve the secure handling of signing keys by supporting expiration and revocation. Expiration means the key specifies a time at which it stops being valid, and once that time comes, signatures made by that key are considered invalid. Revocation similary means the key specifies a time at which it stops being valid.

GET /users/aviva/keys/key1

{ "@context":     "https://w3id.org/security/v1"
, "@id":          "https://example.dev/users/aviva/keys/key1"
, "@type":        "Key"
, "owner":        "https://example.dev/users/aviva"
, "created":      "2019-01-13T11:00:00+0000"
, "expires":      "2021-01-13T11:00:00+0000"
, "publicKeyPem": "-----BEGIN PUBLIC KEY----- ..."
}

Requirement: When verifying a signature, compare expires and revoked, if one of them or both of them are present, to the current time. If at least one of the 2 times is the current time or earlier, then consider the signature invalid. If using a cached version of the key, try to HTTP GET the key and try to authenticate once more, because it's possible the key has been replaced with a new valid one.

(5) Ed25519 actor keys

Allows actor keys to be Ed25519 keys, by allowing the publicKeyPem field to simply contain a PEM encoded Ed25519 public key. The HTTP Signatures draft lists more algorithms; we could support them too. This proposal just suggests that we all start supporting Ed25519 in addition to RSA.

(6) HTTP Signature draft 11

The draft linked above, from April 2019, makes some changes and recommendations. This proposal suggests we adopt them:

For the algorithm parameter, use the value hs2019, or none, and start deprecating the old values (such as rsa-sha256).
The new created and expires parameters seem to be mostly useful to web browser based clients, while our usage of HTTP Signatures is between servers. So perhaps they aren't very useful here. But if someone finds them useful, let's support them.
Support at least Ed25519 in addition to RSA, see proposal A.5 above.

(7) Key rotation using a pair of server-scope keys

Allows to easily and computationally-cheaply perform periodic key rotation.

Rationale:

If you deliver an activity and then rotate the key, the target servers will want to fetch the old key to verify your signatures, but, the old key has been replaced, so they will fail to authenticate your requests. When using per-actor keys, it's possible to try waiting for a time the user is inactive (which is hopefully common because most people probably sleep for a few hours every day), and use that as a safer chance to rotate the key. During the quiet time, other servers will have had enough time to process their activity inbox queues, and by the time we rotate, nobody will want the old key anymore.

The weakness of that solution is that:

It's limited to periods of inactivity, which may limit rotation to once per day or less (what if you want to rotate more often? Hmm is there a good reason to? I'm not sure, just saying hypothetically)
It doesn't work for users that don't have inactivity periods, e.g. a user that uses scheduled activities, automatic responses etc.
It involves the computation of generating a new key for every user every day (assuming we don't want to rotate more often), which I suppose can be somewhat heavy, especially for RSA (but I haven't done any measurements)
It involves lots of network activity because other servers will be fetching the new rotated keys all the time, keys can't be cached for days or weeks or more if they keep being replaced every day or every hour (but I haven't done measurements of the effect on the amount of network requests)

The proposal:

Each server has 2 or more server-scope keys. For simplicity of discussion, let's assume a server has exactly 2 keys, key A and key B.
The server does periodic rotation, but each time, it rotates one of the keys and leaves the other intact. It rotates key A, then next time it rotates key B, next time it rotates key A again, next time it rotates key B again... and so on.
When signing HTTP requests, the server always uses the newer key. For example, if it just rotated key A, it will sign the next requests with key A. When time comes for the next rotation, it will rotate key B and stop using key A, switching to using key B for signing requests.
The time frame suggested here for letting other servers finish processing our activities in their inbox queues is one hour, although this is just a suggestion and open to discussion. So it's suggested you do periodic rotation at most once an hour (or at least leave a key available for at least an hour without change after you stop using it)

That way, when one of the keys is rotated, the other key is still available for another hour and other servers are able to use it to verify the signatures we sent. There's no need to wait for users to be inactive, and it's very cheap: Rotate 1 key per hour. Especially if that key is Ed25519.

(B) ActivityPub

(1) Non-actor audience

(2) Authenticated inbox forwarding

(3) Non-announced following

(4) Object nesting depth

(5) Object capability authorization tokens

(C) ForgeFed

(1) Actors

(2) Authorization and roles

(3) Comments

Comments are Note objects, published using the Create activity. Requirements, suggestions and details:

The AS2 context property must be specified, and must be a single value, and refers to the discussion topic, which is a ticket or a merge request or a patch or something else.
The AS2 inReplyTo property must be provided, and must be a single value, and either specifies the same value as context (which means it's a top-level comment under the topic), or specifies another comment, to which it replies.
When receiving a comment with context C and inReplyTo some existing comment message M, verify that the context of M is C too
Some objects that are discussion topics, such as tickets, have a followers collection, which should include all the previous commenters on the topic (if you comment on a ticket, you probably want to start following it to be notified on new comments, although this isn't required, so that people can opt in and opt out of notifications) as well as anyone who sent a Follow activity even without commenting, as well as the topic author (e.g. ticket author), and this followers collection can be used for comment audience.
Some objects similarly have a team collection (a new proposed ForgeFed property). The team collection would include people who manage the object. While followers is usually opt-in, team is usually opt-out: For example, in a project managed by 3 people, the default could be that all 3 of them get notified on every new comment on every new ticket, but once a ticket is assigned to one of them, the others can opt out of notifications to reduce the noise in their personal inboxes. But how team works is behind-the-scenes for comments: Just consider it to be a collection of team members managing the objects and who want to be notified on new comments.
Some objects that are discussion topics are actors, or are managed by an actor (for example tickets may exist under projects, and projects are actors), and when you deliver the comment to them, they store and display it in the appropriate discussion page
Normally, the audience of a new comment would include:
- The discussion topic (if it's an actor) or the actor that manages it (e.g. the project to which the ticket belongs, on which you're commenting)
- The topic's followers collection
- The topic's team collection
- Specific individuals/groups you want to mention or bring the discussion to their attention
This setup with a followers collection means that clients don't need to do any digging and querying to figure out who the commenters and team members are, and it allows people to opt out of notifications if they previously commented on some topic but don't want to be in the discussion anymore. This makes it very easy for clients to correctly address comments.

GET /luke/outbox/A0O8l

{
    "@context": "https://www.w3.org/ns/activitystreams",
    "id": "https://dev.federated.coop/luke/outbox/A0O8l",
    "type": "Create",
    "to": [
        "https://dev.federated.coop/luke/text-adventure",
        "https://dev.federated.coop/luke/text-adventure/issues/113/followers",
        "https://dev.federated.coop/luke/text-adventure/issues/113/team"
    ],
    "actor": "https://dev.federated.coop/luke",
    "object": {
        "id": "https://dev.federated.coop/luke/comments/L0dRp",
        "type": "Note",
        "attributedTo": "https://dev.federated.coop/luke",
        "context": "https://dev.federated.coop/luke/text-adventure/issues/113",
        "published": "2019-05-26T11:56:50.024267645Z",
        "to": [
            "https://dev.federated.coop/luke/text-adventure",
            "https://dev.federated.coop/luke/text-adventure/issues/113/followers",
            "https://dev.federated.coop/luke/text-adventure/issues/113/team"
        ],
        "content": "That's such a wonderful idea!",
        "inReplyTo": "https://poetry.space/aviva/comments/xN82v"
    }
}

TODO:

followers is for actors, it seems, by the AP spec? So, what do we call the collection of followers of a ticket?
replies and how the C2S object nesting depth proposal
Use nonActors in the example?
Content format? HTML? Markdown source? Tags? Referenced ticktes?
Visibility and privacy?

20 KiB

Raw Blame History

ForgeFed/ActivityPub Federation in Vervis

Federation triggered by regular UI

GET endpoints

POST endpoints

Spec

(A) Authentication

(1) Actor key(s) in a separate document

(2) Multiple actor keys

(3) Server-scope actor key

(4) Actor key expiration and revocation

(5) Ed25519 actor keys

(6) HTTP Signature draft 11

(7) Key rotation using a pair of server-scope keys

(B) ActivityPub

(1) Non-actor audience

(2) Authenticated inbox forwarding

(3) Non-announced following

(4) Object nesting depth

(5) Object capability authorization tokens

(C) ForgeFed

(1) Actors

(2) Authorization and roles

(3) Comments

(4) Tickets

(5) Patches

(6) Merge requests

(7) Commits

(8) Forks

(9) SSH keys

(10) Pushes

(11) Avatars

20 KiB Raw Blame History

ForgeFed/ActivityPub Federation in Vervis

Federation triggered by regular UI

GET endpoints

POST endpoints

Spec

(A) Authentication

(1) Actor key(s) in a separate document

(2) Multiple actor keys

(3) Server-scope actor key

(4) Actor key expiration and revocation

(5) Ed25519 actor keys

(6) HTTP Signature draft 11

(7) Key rotation using a pair of server-scope keys

(B) ActivityPub

(1) Non-actor audience

(2) Authenticated inbox forwarding

(3) Non-announced following

(4) Object nesting depth

(5) Object capability authorization tokens

(C) ForgeFed

(1) Actors

(2) Authorization and roles

(3) Comments

(4) Tickets

(5) Patches

(6) Merge requests

(7) Commits

(8) Forks

(9) SSH keys

(10) Pushes

(11) Avatars

20 KiB

Raw Blame History