Zoom Logo

Secure Data Storage - WG - Shared screen with speaker view
Daniel Buchner
06:26
q+
Andreas Freund
07:40
that would be awesome!
Andreas Freund
07:47
less holy cows!
Manu Sporny
09:19
q+
zokama
10:20
no objection
Dave Longley
10:33
no objection, just appreciation to the editors
Neil Thomson
11:12
ToIP on Portability and Storage WG contacts:Christoph (chair) christoph@ownyourdata.eu
Neil Thomson
13:01
Paul Knowles (Input and semantics Group) - paul.knowles@humancolossus.orgNeil Thomson (ToIP contributor) neil.thomson@queryvision.com
Andreas Freund
13:07
q+
Dmitri Zagidulin
13:39
q+
Derek Trider
13:45
q+
Manu Sporny
14:40
q+
Dmitri Zagidulin
15:28
+1 to separate pagination proposal
Neil Thomson
16:40
Would also suggest a way to return the number of pages in the response (e.g. response: first page, plus number total pages?)
Dmitri Zagidulin
16:54
and we should definitely look to existing pagination mechanisms. Things like 1) JSON-API, 2) CouchDB api, 3) ElasticSearch API, etc
Dave Longley
17:01
vital that there be DoS notes ... asking for too many docs could result in GiBs of data in return... we have good bounds without this feature right now
Dmitri Zagidulin
17:54
q: <empty>
Dave Longley
17:59
q+
Derek Trider
18:36
q+
Dave Longley
19:24
q+
Andreas Freund
20:00
q+
Derek Trider
20:19
Ah, I get what you mean now
Troy Ronda
20:25
Not having to make two calls is better.
Troy Ronda
20:41
I missed the start of the call, apologies.
Dave Longley
21:31
there are workflows where one call that receives 10 docs in serial is slower than 10 calls to receive 10 docs in parallel
Dave Longley
21:39
q+
Troy Ronda
22:12
There are workflows where its faster to make one call too.
Dave Longley
22:50
the fact that we aren't also sending the streams points to the idea that it's not always better to do everything in one call :)
Troy Ronda
23:45
This is different - it needs two sequential calls.
Troy Ronda
23:53
Get the identifier and then make a call to get the document.
Troy Ronda
24:02
Clearly there are cases where not having that extra call is faster.
Dave Longley
24:32
IMO, those cases need to be significantly faster to justify the optionality in the API
Troy Ronda
24:59
We can’t treat everything this way.
Dave Longley
25:24
if the multicall approach could get nearly the same performance with an abstraction in the client -- then that's preferable
Troy Ronda
25:34
It’s clearly slower by a network call.
Dave Longley
25:43
we want to keep the API surface as small as possible -- it makes it easier to implement and mtainain
Troy Ronda
25:52
I want to optimize networks calls.
Troy Ronda
26:05
This is a small difference in the API to drop a network call.
Troy Ronda
26:17
We can’t focus purely on the API surface.
Troy Ronda
26:55
So what’s the outcome with Derek’s PR?
Dave Longley
27:18
I agree that we can't focus purely on the API surface -- I'm just saying we need to justify additional complexity in the core spec (particularly if it could be added in clients instead)
Dmitri Zagidulin
27:33
@Troy - the full document PR? just needs an issue/warning, and it’s good to merge
Derek Trider
27:40
@Troy I will make some changes to it based on feedback I got today
Dave Longley
28:44
(Troy, my point is that we need a strong gating function on adding complexity ... or we will just add *all the things* :) )
Dmitri Zagidulin
28:50
q+
Dave Longley
30:31
tobias -- is another use case to revert changes?
Troy Ronda
30:32
When I see unneeded sequential network roundtrips, I want to optimize those :).
Dave Longley
31:47
sure ... we just may disagree on the necessity/utility in this case (e.g., better security model)
Dmitri Zagidulin
31:56
@Troy - it’s the “sequential” part that I wonder about. (vs in parallel)
Troy Ronda
32:13
Call 1 - Get IDs and wait for response.
Troy Ronda
32:23
Call 2 - Use response to request document.
Troy Ronda
32:26
sequential
Dmitri Zagidulin
32:42
ah I thought you meant the individual document calls in part 2 are sequential
Dmitri Zagidulin
33:09
I do want to point out that this two-step process plays an important role in many document stores (Cassandra, MongoDB, CouchDB, ElasticSearch, etc)
Dmitri Zagidulin
33:26
but each of those also exposes the ‘fullDocument’ flag as well.
Dmitri Zagidulin
34:33
or to put it another way - this is not about optimizing. it’s a straight-up engineering tradeoff. You’re trading longer latency for higher throughput
Dmitri Zagidulin
34:40
and it’s not always a beneficial tradeoff
Dave Longley
35:09
also, a lot of the time you want to *do* something with the documents that come back from that call ... and you may not want to wait for every document to be returned before starting that thing
Dmitri Zagidulin
35:19
^ yep, that
Manu Sporny
35:34
q+
Dave Longley
35:35
in this case, you almost always want to decrypt if nothing else... so you want to be using CPU cycles while you download vs. waiting.
Troy Ronda
36:16
I think it’s important to note that yes DBs have that flag for a reason ;).
Dmitri Zagidulin
36:24
::nods:: agreed
Dmitri Zagidulin
36:35
but they also have a lot of warnings around that flag.
Dave Longley
37:02
and those databases don't do client-side encryption (that i'm aware of)
Derek Trider
39:41
I guess no databases do client-side encryption, technically - EDV just enforces it
Derek Trider
40:00
Something else to note - it it wasn't for CouchDB being able to return more than just the document ID, my EDV server would itself be slower...
Dave Longley
40:57
q+
Derek Trider
41:15
Though I believe supporting the "return full documents" feature shouldn't be too difficult
Derek Trider
41:27
(implementation-wise)
Dave Longley
45:29
q+
Dmitri Zagidulin
47:16
404 or a 410
tobiaslooker
47:57
+1 to Daves response
tobiaslooker
48:01
Out of scope initially
Manu Sporny
50:26
q+
tobiaslooker
51:19
q+
Manu Sporny
52:00
q+
Daniel Buchner
52:14
q+
Dmitri Zagidulin
52:39
are you suuuure? (that those people never meant it to develop this way?)
Dave Longley
52:43
.well-known can cause trouble for multitenancy
Dmitri Zagidulin
53:07
@Dave - that’s only if you don’t give your tenants their own sub-domain. which you absolutely should
Dave Longley
53:22
heh... that is fraught with problems :)
Dmitri Zagidulin
53:41
what do you mean? it’s vastly preferable, security wise, than directory-level multi-tenancy
tobiaslooker
53:42
Ha was just about to make Dmitri’s point :)
Dave Longley
53:42
such as the PSL (public suffix list) + cookie privacy issues that apple and facebook are fighting about right now
Dmitri Zagidulin
53:56
look at the lesson LiveJournal learned, about that
Juan Caballero
54:19
^ tell me more ?
tobiaslooker
54:22
Tenants based on url paths can be a mess too
Dave Longley
54:22
you have to declare your top-level domain a public suffix and lose cookie access if you take that approach
Dmitri Zagidulin
54:42
oh, when LJ started, they gave each user an account in the old-school university style, like example.com/users/~dmitri/
Dmitri Zagidulin
54:49
and immediately ran into cookie security problems
Dmitri Zagidulin
54:59
and moved to subdomains, dmitri.example.com, for their multi-tenancy
tobiaslooker
55:00
q+
Dave Longley
55:01
everything is difficult
Andreas Freund
55:02
q+
Juan Caballero
55:09
oh rt
Dmitri Zagidulin
55:31
q: Tobias, Andreas
Dmitri Zagidulin
55:42
@Dave - ok, but there’s a difference between “difficult” and “completely insecure”.
Dmitri Zagidulin
55:48
which is what non-subdomain multi tenancy is, that latter
Manu Sporny
56:15
/me raises his "burn it with fire torch" in support of Daniel.
Dave Longley
56:26
/me wonders if livejournal is on the PSL
Daniel Buchner
56:42
Put the ref into the Service Endpoints of your DID
Daniel Buchner
56:50
if you really want to make it discoverable
Daniel Buchner
56:55
this is how we will do it with Hubs
Manu Sporny
57:06
q+
Dave Longley
57:32
https://publicsuffix.org/list/ <-- don't see livejournal on there
Juan Caballero
57:42
+1
Dave Longley
58:28
the EDV *API* endpoint should be discoverable.
Dave Longley
58:38
q+
Dave Longley
59:08
s/the EDV/the root EDV/
Dmitri Zagidulin
59:57
I think that issue is just for the root
Andreas Freund
01:00:00
under "services" in a DID doc is another option
Dave Longley
01:00:19
or in a VC that is shared/posted somewhere
Juan Caballero
01:01:53
thx all
Manu Sporny
01:02:59
I'd be fine w/ hogging the weekly call on it if people think it's improtant enough. :)