First things first:
I want to thank V, one of the Zcashers that attended the summit, who lives in Sofia and helped all of us quite a lot to find good places to eat and not fall into tourist bait places and get a real feel of the city.
Now to our chronicle.
The day would start early and with an agenda that will be a challenge for any moderator since the pace of it would leave little wiggle room and one thing I’m sure of is that Zcashers are freethinkers and very opinionated about any Zcash topic. According to the logistic comms sent days prior to the summit the room would be prepared to welcome chatty 34 zebras.
Chatham house rules won’t let me tell you about the round of introductions and who was there but thanks to ZF’s great idea of co-locating the Dev summit with ZkProof, HACS and Real World Crypto we were able to welcome new folks and hear them out. DC was the host and moderator of the first day and set up the rules for the discussion.
The Zcash stack
Just in case you weren’t following the last months of discussions, Zcash is preparing for an “all new” technology stack. Well, partially new because Zebra hour beloved Rust implementation of the Zcash protocol consensus node has been validating Mainnet blocks for quite a while.
Currently the main components of the Zcash tech stack are Zcashd, lightwalletd, Zebra and the different wallets. The node layer is shared both by Zcashd and Zebrad, while the light client server layer is comprised of Zebra’s lightwalletd RPC and lightwalletd itself while Zaino, Zingo’s “winning horse” is catching up fiercely. Lastly, the wallet layer is composed of Zcashd’s wallet, the many shielded wallets and soon Zallet will join the group.
Zcash Z3 is the new stack that will drive the future of the Zcash ecosystem forward. Zebra, Zaino and Zallet will be the backbone of this new era for ZEC. We say new era because developers anticipate that not only will the stack bring a modern, modular, memory-safe technology to work with, they expect that this will also encourage teams to maintain their own copies inaugurating a new time of collaboration and decentralized technical growth.
Zcash Z3 is the new stack that will drive the future of the Zcash ecosystem forward. Zebra, Zaino and Zallet will be the backbone of this new era for ZEC. We say new era because developers anticipate that not only will the stack bring a modern, modular, memory-safe technology to work with, they expect that this will also encourage teams to maintain their own copies inaugurating a new time of collaboration and decentralized technical growth.
What other components are out there?
Miners are the most essential component of our network, they provide network security one block every around 72 seconds at a time. Some of them are already using Zebra Block template methods for their blocks. ZF has one Zebra who is really into grassroots mining and his input has been crucial to improve mining ergonomics of the Zebra API. Zcash has a miner centralization problem that has been a challenge at the settlement layer for users. Whenever a third party like a CEX needs to determine how “final” a given transaction is, they look at the network and see that we have a high concentration of miners belonging to the same pool so they factor this into their coefficients and calculations which often results on users having to wait a long time to withdraw or exchange their ZEC. This needs to be improve if we want to bring ZEC into DeFi.
Speaking of blocks, Explorers are another piece of this puzzle of private and decentralized blockchain economy of ours. They will be the first line of adopters of the ZeeThree stack. The Zcash Block Explorer project definitely needs some love and updates, for which the LCWG has forked NightHawk’s project into the GitHub Zcash repo to accommodate the needed changes to bring ZBE to the new Z3 stack.
The last big component identified by the room was Exchanges: centralized and decentralized. This miner centralization is a huge deal to improve our integration with these actors. Also, they will be the main users of Z3. How to engage with them and outline a strategy that allows Z3 to be deployed without major interruptions (or none) will be discussed later.
What are the pain points?
To the U-shaped bunch of zebras gathering on that meeting room there were basically 4 problems:
- Usability problems
- Syncing and all it brings
- Additional requirements imposed by privacy
- Metadata leaks through the network layer
Zcash shielded Assets
Are ZSAs a big game changer? Asks somebody to the rest of the room. Many say they are and others are dubious. I personally have a professional quirk which is not believe in “silver bullets” so I consider myself between the people that think that ZSAs are great but there is a lot of things to do before they can reach their full potential. I even zoned out a bit while making a full list of what’s needed to do before ZSAs can be launched to discuss later if there was time (there wasn’t) and it was quite long.
One valid criticism to ZSAs is that they were hard to explain to others. A great deal of the success of a product, platform, etc is that its objective can be understood and appreciated by “the masses” and this Zebra saying “Zcash is better than Monero in tech but they are still better than ZEC when it comes to adoption” which resonated across the room and someone said “I struggle to explain Zcash in a brief time, with miners you can simply explaining saying ‘decoys’”.
Tailoring the explanation to the audience is a challenge in every field. Moreover in crypto where best case scenario the person on the other side of the table doesn’t know anything about it or (worse) it has misconceptions that need to be debunked before proceeding with the explanation. ZSAs are yet another hurdle for the Zcash evangelist. Another complication that other crypto folks don’t have.
Ok, enough ranting. The community is very enthusiastic about ZSAs, but we don’t have to focus solely on that.The other part of ZSAs are issuers. Who will be minting these assets that the community wants? Do they have all that they need to actually do it? What does an issuer’s treasury require to function?
That question can only be answered by those who actually run one and apparently some members of the Zcash community had already been in touch with that kind of entities to get that information.
We have to be Decentralized but Organized! there are a lot of things to do and many of those not depend solely on us Zcashers but on third parties as well. That’s why I think that it’s important that ZSAs have launch partners because it’s unrealistic to think that we can both shoot a long pass and also jump and dunk the ball to score this goal. That phrase resounded. Not only because how catchy it is, but also because it vibrated at the same frequency of the room, which also brought up fragments of Zaki Manian's talk on ZconVI
Going back on the notes of other past summits that had been held by Zcash organizations, there was an idea that came up in at least two of them that I can remember: “Make Zcash Fun” and “why can’t we have nice things?”. This is actually something that had been said in the Zcash Forums by Taylor Hornby while back “stop drama” was a powerful message by him. There is a history of drama between two of the major Zcash organizations which derived in people rage quitting, gatherings that featured heated up “dialogues”, sudden livestream panel interruptions, unfounded and accusatory threads in the forums and other things that are not worth enumerating but yet existed and that don’t make us proud but are part of the “lore” of the Zcash community.
I’m very happy to say that the vibe in the air was very different this time.
Decentralized but Organized is the catchphrase that not only condensed the energy of ZF’s Zcash Dev Summit but was inspired by it. People putting aside any past differences and looking forward to what it yet to be built and the future that we as Zcash developers, have the responsibility to build because we do know that the community is demanding it and we owe them that.
In order to achieve this Decentralized but Organized flow, there has to be a clear vision of the different layers of this endeavor. There are parts that can and will be done in a decentralized fashion as the concurrent development of Zebra, Zaino and Zallet but others will be inherently more centralized like for example all the work that is not clearly been accounted for QEDIT to merger all of their work upstream. Those repositories (unfortunately) have a centralized set of maintainers that we haven’t been able to fully decentralize yet.
WHEN ZSA is a question that floated around the place for both days. I think that it’s a fair question for the community to ask but also a difficult one to answer precisely. “The most important thing is to deprecate Zcashd” and heads nodded across the room. There are other things that are not precisely know. Somebody hit the spot when asked “do we have an idea on who the users are for ZSAs?”. The when question increased the nervousness of the participants of the debate, even a more extremist on suggested that “there should be a benevolent dictator taking over for a year to get us through all of this” but that clearly did not have consensus yet, it did help clear the air into more constructive thoughts.
“There is no queue of people asking us to launch ZSAs or anything in Zcash, in fact nobody is using Zcash that much” - Someone said, while other counter argued: “every investor I talked to was super pumped about ZSAs when they learned about it, they did not have any idea they were a thing”. And there was a complementary view that added “we lack understanding and acceptances of why other platforms are successful”. We do know that “people will compromise their privacy in favor of usability” duly noted someone with a mix of experience but also frustration.
How ZSAs are deployed and made available to the world matters a lot. Remember: people will compromise privacy and other aspects of their self-sovereignty over usability. So to help them regain that relinquished self-sovereignty we need to provide them tools that are both available and usable to them. Protocol capabilities are not enough on their own. Going back to the Zcash Lore, some concrete cases were brought up. First, Flyclient, an addition to the Zcash protocol and compact-block model that would have allowed desktop and mobile wallets to have better verifiability of their transactions but no-one actually built it, neither the original authors of the paper nor the ECC team who was in charge the light client infrastructure development at the time. The second case was Unified Addresses and the lack of readily tooling that allowed their adoption. UA parsing tools are still lacking nowadays more than 4 years after their deployment. UAs are currently not able to solve the problem they were destined to and moreover, contributed to make it even worse by adding yet another address type instead of being the “one type to rule them all”.
We have enough lessons in our history that teach us that launching protocol features on their own is not enough and doesn’t add value unless all the additional structure that makes the feature be accessible to users is also available at launch.
I’m seriously doubting on releasing my “off-the-cuff ZSA list of pending things” but still it’s a good exercise for everyone to think about that write in on a piece of paper.
Zcash Proof of Stake
Hours went by quicker that I thought they would be and it was time to dedicate some portion of the day to the next next big thing: Hybrid Pow/PoS. It was great to have one member of shielded labs with us plus other important members of the community that were pretty knowledgeable on PoS.
The biggest precedent of a PoW chain going PoS is Ethereal and they did it for several different reasons. The room concluded that the main reason for Zcash is Security. Protocol experts in the room explained others how PoW has failed to achieve the security goals stated on the original Bitcoin paper. Probabilistic finality is not enough for modern DeFi.** Wait, what’s that?**
Proof of Work mechanics are noble and simple: whomever has the most work proven decides what the next blocks. That means that whomever accrues the most proofs-of-work has decision power over the rest of the network which makes it possible to assert that there is a probability p, no matter how small it is that a miner or a set of miners can decide and achieve arbitrary rollbacks of the chain by leveraging the sacred rules of the Nakamoto consensus. Meaning, that PoW has probabilistic finality. Transactions are considered “final” by someone given that the probability of them being reverted is acceptably small to their criteria.
DeFi and moreover privacy over it needs fast finality
To know a more detailed view of a very important portion of what is going on to be discussed onwards go to https://shieldedlabs.net/blog
Shielded Labs is developing Crosslink 2, a mechanism to combine two view of the same chain validated by two different consensus algorithms. Crosslink 2 is a means to an end. It is not necessary to run a Hybrid PoWPoS mechanism forever, but still POS will have to be validated and supported by the Zcash community.
Within the next 18 to 24 months Shielded Labs proposes to develop and deploy the crosslink version of Zebra which will bring fast finality to our formerly PoW only network. FF will not necessarily mean faster block times but it will definitely mean that people won’t have to wait onerous delays to have their ZEC transactions considered settled by others like CEX, DEXs or Bridges.
PoS and recovering a lost opportunity
Can PoS be a way to bring grassroots enthusiasm back to Zcash?
The whole room recalled a situation that some years ago, when the ASIC miners hit the market the Zcash developer community had to make a decision: either to play “cat and mouse” game and constantly develop new solution algorithms that make dedicated hardware like ASIC inefficient until someone figured it out, or just embrace the new crypto reality of the dedicated hardware industry that was rising. That decision came at a price, many “home grown” Zcash miners left the community searching for other horizons. “I mined ZEC with my GPUs until ASIC came” is a repeated story I hear from anonymous Zodlers. They understandable departure is something most of the room regretted but also admitted that modifying the solution algorithm and hashing functions to achieve ASIC resistance has its own risks and even when they were evaluated as “worth having” it was not guaranteed that one morning an ASIC miner wouldn’t develop the ability to mine ZEC.
Back to the original question, the answer is YES!!
PoS can help achieve that sense of grassroots participation in network security that was lost to ASIC miners. By allowing people to set up validators with the minimum requirements as possible and penalizing aggregation of staking, the network would require that many validators are created to secure the network. The other important topic is delegation. On one hand, staking delegation allows small ZEC holders to stake their holdings and earn some reward for their stake contribution. But something important that protocol experts debunked is whether those small delegations actually contribute to decentralize the network or not.
In my experience playing around with other tokens, although delegating stake helps to create a sense of token utility and can lower selling pressure, it doesn’t feel like contributing to network security that much. Taking Solana as an example, when you buy some SOL tokens you can use the Solflare wallet to browse a validator catalog that gives you basic information about the validator itself, fees and percentage of staking rewards. The UI tells you “don’t choose validators that have more than 30% of the stake to help centralization”, after making your choice, you leave your tokens staked and that’s it. This might not be super accurate, but that’s what I recall I did many years ago when I had curiosity about Proof of Stake and stake delegation. I don’t know whether it changed or not. I actually didn’t changed anything about those staked SOL. Did I participate in network security? What are your thoughts about it? To me, It doesn’t feel I did.
That’s why someone said “Delegation shouldn’t be allowed or no one will run validators, they will all delegate to centralized validators.”
That’s why this question is so relevant: “How do we think staking will be? Are we thinking about how the staking protocol will align with what we think Zcash values are?”
Because staking doesn’t come without its setbacks. For example, Zcash’s commitment to Satoshi’s vision of 21M ZEC supply cap. “There’s a problem with staking and earning revenue and a supply cap. No other PoS implementation has a fixed supply”. This is no small detail. Are staking rewards inflationary? How will that affect ZEC tokenomics? Those are open questions the Zcash Ecosystem needs to start asking and looking for answers as well. Another problem that was brought up by more experienced Stakers in the room is the case of “0% fees” validators in the Cosmos ecosystem which end up capturing the vast majority of the delegated stake of the network leveraging people’s individual financial needs and this is detrimental to the network stability and security when the original goal was exactly the opposite.
A few questions remained before the next section of the day began. What decisions can be made later and just pick what works out of the box now? On the other hand, what are the things that need to be decided now so they don’t irreparably break in the future?
Zcash Scalability
If you haven’t watched Sean’s talk at Zcon VI, please do so now. We will tap into the concepts during this section of the day.
What model is best for scaling Zcash? Sean’s take on this is that outsourcing the cost of syncing the network with recursive proofs is they best way to achieve a scalable Zcash that doesn’t compromise any of the privacy guarantees that the protocol already has.
Sean proposes that Oblivious Syncing and maintenance of a Nullifier set can be delegated to untrusted servers. These servers don’t exist today and someone will have to run them. “Who in the ecosystem will be the best suited to run this infrastructure? Would it be ZingoLabs, Shielded Labs, ZecRocks?” Moreover, “can these service providers safely rely on users paying for it or shall this be embedded into the protocol?”. But there was rebuttal across the room. “The problem of this is that we are solving all scalability problems by delegating a lot of things to a third-party service that doesn’t yet exist”. This concern is very valid. Also, making Zcash scalable means (according to what was explained) changing assumptions of how Zcash’s blockchain works now and would work in the many years to come.
As time passes the blockchain will grow one block at a time. To users, a scalable Zcash means having the ability to have full blocks without feeling the effects that they experienced during “sandblasting”. There is no magical way to make the blockchain lighter. There are different strategies to achieve this. Off-band information is a way to actually make the blockchain lighter by relying on the wallets to handle some information. “Off-band payments could have already been done by liberated payments” someone said. For some reason, Liberated Payments come up in every single developer meet up I’ve attended as a great thing but still no team has effectively pursued their complete implementation and deployment. That’s something that catches my attention.
Another way of making blockchain protocol lighter is by pruning the information that nodes or clients that need to provide certain features of the protocol. But what does that mean to Zcash? “Can a scalable Zcash continue to provide a blockchain that can let a user post an address on a billboard and expect that address to use it as a storage layer until the end of time?”.
The answer came from the other side of the room: “Scalable Zcash means that not all use cases would be possible...”
And was not received well. It outraged some of the people.
“if I have one ZEC I expect it to have it 10 years from now! Otherwise we should advertise that”
Communication is the most difficult part of any human endeavor. Some people interpreted that this would mean that if they had ZEC in an address and they buried those keys in the ground and dug it out eight or ten years later, their ZEC wouldn’t be there.** Which it not the case**.
But also, can any system be evolved in time maintaining 100% full retro-compatibility for all of its past features? That’s an interesting question in terms of software engineering. Blockchain are these “forever machines” that happen to be pretty new. Bitcoin is about to reach its legal driving age for many countries, not even majority of age (18-yo).
“My seed phrase should be the only thing I should need to recover my funds” said somebody. Yes… but…. “Wallet.dat” does it ring any bells? Zcash currently has a team working on something called “Zmigrate” which is a piece of software that would help people that had node running years ago, can now collect the information of that old node wallet and bring it up-to-date with latest protocol developments by translating that file into a Wallet information format called ZeWIF. So this is already something that happens to people holding Sprout funds. They need to do some additional effort to access their funds, but the Zcash blockchain still guards them.
So there appears to be two opposing forces: long term hodling and protocol advancement. And I believe these forces are inherently natural in a “perpetual engine” like the Zcash blockchain. So, how shall Zcash developers balance these forces? How can they secure long term zodling while still evolving the protocol to guarantee that the Zcash supply is secure from adversarial technological advancements?
There was an idea proposed by ZF’s former Engineering Director at the inaugural talk of Zcon VI where the proposal of Zcash sidechains were introduced.
Someone proposed that “Maybe there should be another chain, layer or pool designed for long term storage” of funds where “this long term fund storage pool could be the pool that is used for staking” but this had a very good “reality check” counter-argument. “We assume that people are more technical than they are”. This is a very good wrap-up for ZF dev summit day 1.
Day 2 was a collaboration day between all the teams present and a very hands-on morning and afternoon which didn’t leave me much space for note taking.
I'll leave you all with some photos of the beautiful Sofia and the events many of us attended during the week
