Zoom Logo

Secure Data Storage - WG - Shared screen with speaker view
Dmitri Zagidulin
01:41
attendance/proposals doc: https://hackmd.io/vUTZ2DdpQ6GxQz9YTPk1CA
Dmitri Zagidulin
03:19
attendance doc: https://hackmd.io/vUTZ2DdpQ6GxQz9YTPk1CA
tobiaslooker
03:34
https://lists.identity.foundation/g/sds-wg/wiki
Manu Sporny
04:02
/me misheard that as "please refer to our Wookie"... and I got really excited there for a second.
Dmitri Zagidulin
04:41
alas :)
Manu Sporny
04:47
Welcome, Vaner!!! :)
Vaner Vendramini
05:12
Thanks!
Michael Shea
05:43
Welcome Leah!
Manu Sporny
05:50
Welcome Leah :)
Chris Were
05:58
Welcome Leah, great to have you here.
Leah Houston
05:59
🙏🏽
Leah Houston
06:23
Leah@hpec.io for those that want to stay in touch
Leah Houston
06:31
yes that was me:)
Orie Steele (Transmute)
08:52
Nothing like getting canceled by twitter to convince you that replication is valuable.
Manu Sporny
09:05
q+ to suggest simplest replication use case - uni-directional backup and why we want it -- wallet backup / portability.
Daniel Buchner
10:01
Active replication - don't want to go to sleep at night and wake up like "Let me do my ghetto manual sync now that I am awake - oops, they deleted me while I was unconscious"
Chris Were
10:12
I’m not sure if people saw, but I made an attempt to outline a range of specific use cases, including complex ones here: https://github.com/decentralized-identity/confidential-storage/issues/161
Orie Steele (Transmute)
10:26
q+
Adrian Gropper
11:06
+1 to simplest
Adrian Gropper
11:36
q+
Adrian Gropper
11:49
to ask about “somewhere else”?
Dave Longley
12:33
"I have one EDV full of data at one service provider and I would like the data copied to another EDV at another service provider."
Daniel Buchner
12:35
Simple backup is an important, but rather limited and doesn't support the 99% of app use cases
Dmitri Zagidulin
12:39
thanks chris!
Daniel Buchner
12:46
Is *important
Manu Sporny
13:34
/me puts the weight of Orie's heavy hint at 900lbs. :P
Orie Steele (Transmute)
13:58
:)
Orie Steele (Transmute)
14:13
Our public implementation is experiemental
Orie Steele (Transmute)
14:17
But it is live
Daniel Buchner
15:06
I just want replication so sweet it pushes message board objects to peers rapidly and proactively, so that the peers listening can create a r/WallStreetBets board that can't be feasibly shut down
Manu Sporny
15:36
q+ to note "somewhere else" as another EDV.
Orie Steele (Transmute)
16:12
In a world where there was at least 1 standard for storing encrypted data associated with dids….
Manu Sporny
17:06
q+
Orie Steele (Transmute)
17:07
Yes, could be 2 separate business
Orie Steele (Transmute)
17:58
Question not clear
Manu Sporny
19:18
q+
Orie Steele (Transmute)
19:54
We should keep poking it until there is an issue that is ready for a PR :)
Orie Steele (Transmute)
20:11
And probably avoid talking about anything that is not an issue.
Chris Were
20:25
q+
Vaner Vendramini
20:32
q+
Chris Were
20:48
Q-
Michael Herman (Trusted Digital Web)
21:09
Can you repost the link?
tobiaslooker
21:22
Which one sorry Michael?
Chris Were
21:35
q+
Dmitri Zagidulin
21:48
q+ to reply
Chris Were
22:04
You go first Dmitri
Orie Steele (Transmute)
22:05
Q: Chris
Orie Steele (Transmute)
22:16
Q: Dmitry, Chris
Orie Steele (Transmute)
22:36
https://github.com/decentralized-identity/confidential-storage
Michael Herman (Trusted Digital Web)
23:07
For the use cases?
Orie Steele (Transmute)
23:21
https://github.com/decentralized-identity/confidential-storage/blob/master/use-cases/index.html
Orie Steele (Transmute)
23:31
https://identity.foundation/confidential-storage/use-cases/
Orie Steele (Transmute)
23:47
Q: empty
Manu Sporny
24:40
q+ to note why we're taking this approach -- we tried "all use cases approach", and end up getting mired in details for every possibility.
Orie Steele (Transmute)
24:49
Q: manu
Orie Steele (Transmute)
25:44
My use case is cold fusion running on my watch
Orie Steele (Transmute)
25:52
Says Daniel ^
Dave Longley
25:53
we are too good at creating incredible use cases ... which makes it too hard to focus :)
Orie Steele (Transmute)
25:53
;P
Daniel Buchner
26:04
I don't think active/active replication requires a complex use case to justify
Orie Steele (Transmute)
26:21
“I think cold fusion is easy"
Orie Steele (Transmute)
26:24
:)
Dmitri Zagidulin
26:36
hehehe +1 orie
Michael Herman (Trusted Digital Web)
26:52
q+
Chris Were
26:54
Thanks Manu. Fingers crossed :)
Orie Steele (Transmute)
27:02
Q: Michael
Daniel Buchner
27:22
I guess the countless other systems that have active/active replication have solved cold fusion then
Orie Steele (Transmute)
27:57
“Other people’s fusion reactors seem to be working, even though I don’t have one”
Daniel Buchner
28:50
Track the change: https://github.com/csuwildcat/syncro/blob/main/main.mjs#L69
Dmitri Zagidulin
29:05
w+
Dmitri Zagidulin
29:11
q
Manu Sporny
29:17
Dmitri is on the Wueue
Orie Steele (Transmute)
29:19
Q: empty
Manu Sporny
29:35
... pronounced Woo!
Daniel Buchner
29:49
Diagnose if out of sync: https://github.com/csuwildcat/syncro/blob/main/main.mjs#L81
Orie Steele (Transmute)
29:56
Daniel keep going you will probably get to OrbitDB in a year or 2
Orie Steele (Transmute)
30:07
https://github.com/orbitdb/orbit-db
Daniel Buchner
30:08
Get only necessary change records: https://github.com/csuwildcat/syncro/blob/main/main.mjs#L102
Orie Steele (Transmute)
30:16
“OrbitDB is a serverless, distributed, peer-to-peer database. OrbitDB uses IPFS as its data storage and IPFS Pubsub to automatically sync databases with peers. It's an eventually consistent database that uses CRDTs for conflict-free database merges making OrbitDB an excellent choice for decentralized apps (dApps), blockchain applications and offline-first web applications.”
Manu Sporny
30:21
My favorite part about this call is the Daniel<->Orie banter.
Orie Steele (Transmute)
30:23
https://github.com/orbitdb/orbit-db
Daniel Buchner
30:26
Orbit doesn't do this - already talked to them
Daniel Buchner
30:39
Already talked to Ceramic too
Orie Steele (Transmute)
30:42
Why didn’t you add the feature for them?
Daniel Buchner
31:06
because their datastore doesn't have the encryption/authz features either
Orie Steele (Transmute)
31:10
Hmm 99% of what I need… better start from scratch.
Daniel Buchner
31:17
every one of these projects has like 60%
Chris Were
31:18
OrbitDB is a really poor solution in this space.
Michael Herman (Trusted Digital Web)
31:20
q+
Chris Were
31:23
Anything IPFS based.
Chris Were
31:52
@Daniel I’d be interested in an article Ceramic v Sidetree sometime ;)
Manu Sporny
32:09
q+ to note "keeping track of them" would be something we'd have to put in the spec.
Daniel Buchner
32:29
Chris: because they aren't a globally sync'd system, they don't have to operate under the same constraints Sidetree does
Michael Herman (Trusted Digital Web)
32:43
q+
Daniel Buchner
32:46
but they do use a blockchain to anchor isolated ceramic datasets
Daniel Buchner
32:54
it's a rather different problem
Vaner Vendramini
33:16
q+
Daniel Buchner
33:29
I have half the logic in a generic 100 LOC module
Daniel Buchner
33:40
which is including actual DB integration
Daniel Buchner
33:45
but I digress
Orie Steele (Transmute)
33:47
Ceramic seems as committed to moving the price of ETH as Microsoft is to moving the price of Bitcoin.
Daniel Buchner
34:17
I am committed to lowering the cost of disempowering governments
Daniel Buchner
34:27
to be honest
Dmitri Zagidulin
35:43
q+
Dave Longley
35:54
q+
Adrian Gropper
35:57
q+
Orie Steele (Transmute)
36:10
I am obviously trolling to a certain degree… I expect the Filecoin ICO will eventually solve this problem for us.
Orie Steele (Transmute)
36:21
Q: Dmitry, Dave, Adrian
Michael Herman (Trusted Digital Web)
37:31
q+ (I might have been missed)
Orie Steele (Transmute)
37:32
SAMPLE IMPLEMENTATION
Orie Steele (Transmute)
38:04
Q: Michael, Dave, Adrian
Daniel Buchner
38:41
If you do this right, you don't have to ask this "Which side do we pick" question
Orie Steele (Transmute)
38:57
Daniel feel free to q
Daniel Buchner
39:04
I linked to the general scheme, like 50-60% implementted
Daniel Buchner
39:10
and it is agnostic to that question
Daniel Buchner
39:14
q+
Orie Steele (Transmute)
39:25
Q: Dave, Adrian,daniel
tobiaslooker
39:56
Charter can be found referenced here https://lists.identity.foundation/g/sds-wg/wiki
Manu Sporny
39:58
The second thing -- we are building a specification that can be implemented.
Orie Steele (Transmute)
40:05
Charter: https://drive.google.com/file/d/1vf2CsD9QZstzrd6CJ4WFVHw0WKwwNLHf/view
Manu Sporny
40:09
This is not an architecture-only specification.
Orie Steele (Transmute)
40:26
See also the readme: https://github.com/decentralized-identity/confidential-storage
Orie Steele (Transmute)
40:47
Q: Adrian, Daniel
Michael Herman (Trusted Digital Web)
41:30
The charter doesn't directly answer my question about the expectations of the specification
Chris Were
41:36
On that basis are we saying the current client -> server “replication” is sufficient?
Manu Sporny
41:40
Michael - answer above
Orie Steele (Transmute)
41:52
We are committed to a spec and support for an http based sample implementation
Manu Sporny
41:52
q+ to answer Michael
Michael Herman (Trusted Digital Web)
42:03
Create one or more specifications to establish a foundational layer for secure data storage (including personal data), specifically data models for storage and transport, syntax, data at rest protection, CRUD HTTP API, access control, synchronization, and a minimum viable HTTP-based interface compatible with W3C DIDs/VCs.
Michael Herman (Trusted Digital Web)
42:14
...from the charter
Orie Steele (Transmute)
42:28
Q: Daniel, Manu
Orie Steele (Transmute)
43:02
:)
Dave Longley
43:10
Will there or will there not be additional API surface at an EDV server for configuring a replication service (nevermind the sort of diff/history APIs required to *enable* replication itself)?
Manu Sporny
43:18
q+ to speak to "do we need to talk about client-based replication?"
Michael Herman (Trusted Digital Web)
44:07
We're still talking about replication as an atomic process …
Dave Longley
44:55
We can talk about this in terms of the APIs we need/don't need: 1. An API enable doing things with history/diffing. 2. An API to configure a replication service that the server provides (that will, in turn, use the APIs from 1).
Chris Were
44:59
Aren’t we supposed to be talking about the simplest use case?
Dmitri Zagidulin
45:11
q+
Chris Were
45:12
@Michael: I’d be interested in your write up.
Orie Steele (Transmute)
45:22
Q: Manu, Dmitry
Orie Steele (Transmute)
46:19
Today, we have 3+ EDV implementations…. That I know of
Michael Herman (Trusted Digital Web)
46:25
Implementable and interoperable across vendor/projects?
Orie Steele (Transmute)
46:32
^ correct
Michael Herman (Trusted Digital Web)
46:41
:-)
Dave Longley
46:51
yes, independent implementations required
Daniel Buchner
47:43
Why do EDVs not live on the client?
Leah Houston
47:50
that actually makes total sense to me....
Daniel Buchner
47:52
I don't see a huge distinction htere
Vaner Vendramini
47:55
I agree with Manu
Dave Longley
48:04
i worry that server-to-server is going to get confusing ... because what really happens in that scenario is one of the servers plays the *role* of client ... what is different is whether or not the data is decrypted/reencrypted.
Manu Sporny
48:05
/me will write a proposal
Daniel Buchner
48:06
it's instances replicating to instances
Daniel Buchner
48:31
the only difference is that some instances happen to have available URLs you can hit
Dave Longley
48:52
q+
Manu Sporny
48:54
PROPOSAL: Focus on and specify server-to-server replication first. No need to specify client-based replication yet.
Leah Houston
48:58
why not.
Orie Steele (Transmute)
48:59
+1
Vaner Vendramini
49:10
Daniel: “i worry that server-to-server is going to get confusing” totally agree
Adrian Gropper
49:16
+1
Michael Herman (Trusted Digital Web)
49:18
-1
Michael Shea
49:18
+1
Daniel Buchner
49:20
q+
Chris Were
49:28
-1
Daniel Buchner
49:30
No idea why we need to talk about these things like client/server
Manu Sporny
49:33
+1
Daniel Buchner
49:33
all instances
Daniel Buchner
49:39
some have URLs to ping globally
Orie Steele (Transmute)
49:45
Daniel, the charter is about http
Daniel Buchner
49:48
some are on phones and have to do hops
Leah Houston
49:49
I don’t understand the question
Adrian Gropper
49:50
q+ to ask about Daniel’s “cold fusion”
Orie Steele (Transmute)
49:50
Thats why we say the words “server"
tobiaslooker
49:51
q+
Manu Sporny
49:53
We do not want to do always decrypt/reencrypt
Orie Steele (Transmute)
50:05
Q: tobias
Dmitri Zagidulin
50:28
q+
Manu Sporny
50:35
EDVs - AOL Dial-in Edition
Orie Steele (Transmute)
50:38
Q: Tobias, Dmitry
Dave Longley
50:46
+1 for replication that does not require decryption
Vaner Vendramini
51:06
+1 for replication that does not require decryption
Manu Sporny
51:12
+1 for replication that does not require decryption
Orie Steele (Transmute)
51:15
+1 for replication that does not require decryption
Daniel Buchner
51:21
Decryption is data sync
Daniel Buchner
51:26
that is not object replication
Daniel Buchner
51:30
it is far more, and at a lower level
Daniel Buchner
51:38
exactly
Daniel Buchner
51:41
what Tobias said
Daniel Buchner
52:34
The module I am writing does efficient object replication agnostic of client/server role - it does no decryption or object convergence sync
Dave Longley
52:35
I agree that smart conflict resolution requires decryption
Orie Steele (Transmute)
52:53
Q: Adrian, Dmitry
tobiaslooker
52:56
Could we agree that replication perhaps does not need decryption
tobiaslooker
52:59
But sync does?
Daniel Buchner
53:29
Replication absolutely does not need decryption
Orie Steele (Transmute)
53:31
Daniel please q to reply
Daniel Buchner
53:34
q+
Manu Sporny
53:35
+1, replication does not need decryption
Dave Longley
53:40
+1
Orie Steele (Transmute)
53:43
Q: Dmitry, Daniel
Chris Were
53:48
q+
Orie Steele (Transmute)
53:56
Q: Dmitry, Daniel, Chris
Leah Houston
54:02
can you briefly explain what that means in layman terms
Manu Sporny
54:17
Leah -- what specifically? Happy to try...
Dave Longley
54:30
A server is going to play the role of client to replicate to another server ... when doing "server-to-server" replication.
Leah Houston
54:50
vs sync
Orie Steele (Transmute)
55:02
Q: chris
Dmitri Zagidulin
55:26
leah - we have a defn in the terminology section
Leah Houston
55:31
it’s the same as interoperability then... no?
Dmitri Zagidulin
55:35
can someone link?
Orie Steele (Transmute)
55:58
https://github.com/decentralized-identity/confidential-storage/blob/05d600a8bf828674a9cd9205cdb519157c6da1cb/use-cases/index.html#L220
Leah Houston
56:22
👍🏽
Leah Houston
56:26
🙏🏽
Orie Steele (Transmute)
56:26
What Daniel is saying regarding merging
Orie Steele (Transmute)
56:29
Is the cold fusion
Leah Houston
56:29
yes. wu
Leah Houston
56:33
will also read
Daniel Buchner
57:16
It depends on what the definition of IS is
Orie Steele (Transmute)
57:37
Required reading, AutoMerge, CRDTs, CouchDB, Kafka,
Manu Sporny
57:40
Define /depends/, Daniel :P
Daniel Buchner
57:47
Diapers?
Kaliya Identity Woman
58:24
+1
Manu Sporny
58:30
+1 to support simplest backup use case in the spec
Orie Steele (Transmute)
59:38
Please use github
Orie Steele (Transmute)
59:38
https://github.com/decentralized-identity/confidential-storage
Orie Steele (Transmute)
59:45
Here is the spec
Orie Steele (Transmute)
59:45
https://identity.foundation/confidential-storage/
Daniel Buchner
01:00:11
Kaliya on the q
Michael Herman (Trusted Digital Web)
01:00:41
Are these links going into the minutes?
Daniel Buchner
01:00:56
I thought a 1 looked like a q, because apparently I am blind
Orie Steele (Transmute)
01:00:56
I don’t know
Dave Longley
01:01:07
if anyone isn't busy, they could analyze all the popular existing replication protocols and indicate the common data model elements we need to support them :)
Michael Herman (Trusted Digital Web)
01:01:19
thk u
Daniel Buchner
01:01:24
Dave: unfortunately I have
Chris Were
01:01:31
Thanks everyone
tobiaslooker
01:01:36
Thanks all