Sonm Roadmap v6
Yes, a distributed computing project also has a mission. The amount of Big Data grows faster than network throughput, it becomes impossible to transfer all information collected from IoT devices to data centers, which leaves us with a question – where that excessing data should be stored and processed? On places, near where it was generated – there are powerful consumer computer around, we just need to reach them.
There are wonderful existing cloud computing techniques and software. OpenStack cloud management, Kubernetes orchestration, Ceph file cluster, Hadoop and Spark to lookup bid data pools. All the mentioned software has something in common – it is designed to work inside a local area network, it is highly dependent on very fast local networking (when you have both fast response and high throughput).
We see our mission as:
Unleash computing fog, protect it and help it grow until it tells us it wants to live on its own.
Goal 1 – gain early platform adoption and grow mass
Large-scale systems are complicated and must be built from individual components, each of them has its own complexity. Components should be preferably tested independently, if the components can be used individually as well – that is even better.
Our mission is long term and can’t be fulfilled in one big step. Instead, we see it as a path and walking along it requires us to make small steps, one at a time. Each of our steps on a path to fog computing platform is a separate component (subsystem), which has its own meaning and it works like a finished floor of a skyscraper under construction.
We see it as building an ultimate skyscraper – first, we need to do the research and prepare the ground, then we make the fundament, then underground levels and parking, then ground floor, then 1st floor and then levels above. One year since our skyscraper ground floor is open for offices, shops, and warehouses, at the time of writing roadmap version 6 we already have a building plan for many levels above the ground. We will explain this plan in the text below.
We build fog platform with one subsystem at a time. We arrange our architecture and software development plans in such a way, that each subsystem, once it is finished, can be used on its own and together with other subsystems as a whole.
This fog computing future will for sure have multiple users and multiple hardware suppliers. The exact algorithms and software that manages the data and runs computations in parallel – may vary (it may be orchestrated cluster, a grid system or something else), but whatever be it – this software should work on top of some kind of shared PC economics. Individual people and companies will contribute their computers to the Fog. These contributors (suppliers) expect to have their wages.
This shared PC economics is exactly the building piece we created so far. While we work on the next stages of Fog Computing platform, this shared PC may already start working on its own. It is not only an opportunity but a must, that our first floor has to be commercially adopted. By starting to use the platform today with its existing PC sharing capabilities:
- we gain users, that create the demand;
- having demand we can keep hardware suppliers on the platform being able to pay their wages;
- we are able to discover flaws and fix them;
- and to gain mass.
By the time we will introduce true fog-specific computing layer (which we plan), there would be enough existing hardware to run on and users to offer services to.
Goal 2 – keep the system distributed starting from its design schematics
Another important goal is to keep the system distributed since its design schematics.
It is true that most of the existing software and services run as centralized authorities, for example reading this text you are reading it from a centralized server. It is a big mistake to think that someone would always prefer a distributed solution just because it is distributed. This is a common reason many blockchain startups fail or are not being used.
It is also true, that given all the same capabilities, people will prefer open source FOSS software. People will also prefer distributed something over centralized give all other features being equal or at least comparable.
That said, we believe it is important to keep the system:
- usable and beneficial in the first place – it must be a nice product first;
- distributed as a final and ultimate competitive argument – it is not only good but belongs to no one, this is a common good.
Sonm will always stay distributed in its core. We already have:
- distributed hardware suppliers pool – anyone can join and contribute;
- distributed marketplace business logic – our marketplace is the first of its kind working computing power as a commodity distributed exchange (commodity DEX).
Notably, we have some components, that work in a centralized fashion today:
- Sonm sidechain uses PoA today – there is a plan to switch to community-powered PoS.
- Sonm has “municipal” software components, the ones we host on our side – data warehouse (DWH) and sidechain sync nodes – these components are all open sourced and our software is able to connect to a custom DHW or use a custom sync node. It is up to the user to establish his own DWH and sidechain sync node, our municipal nodes are provided for convenience and allow our client software to have “light” mode of operation.
- KYC service is provided using centralized authority. This issue can’t be fixed today, but on the platform level there can be many KYC providers, thus multiple authorities, keeping the whole design distributed.
- Trusted Platform feature requires a trusted central authority, and we plan to provide one of a kind. It is possible for others to provide alternative solutions, keeping the general system design distributed.
- There is a rendering service build by Sonm team (called “Rays”). It is a centralized commercial service with proprietary software. However, Rays is merely one of the possible users of Sonm platform, so it does not break the distributed design. One of many possible is ok.
Goal 3 – create a novel fog computing architecture that will allow fog-native software to work seamlessly across the globe
Someone will finally create it – a fog computing architecture, a living fog. It will definitely start with PC sharing, which we already have. It will require a security solution, one that we work on. It will have distributed nature, a sphere we are pioneers in.
We estimate that we have a fair chance to be one of the leaders here.
Sonm as a company has matrix staff structure, having multiple small projects and team members being staffed to one of them (or being shared across many projects). These projects have their own tracks and sometimes independent budgets.
Commercial adoption track
|✅ Rays closed beta||Done|
|🏗 Rays official launch||On the final stage of development.
Active beta testing phase.
To be launched at the end of July 2019.
|👥 App dev team
👥 Frontend dev team
|🚩 Rays long term big stake||Scheduled|
|🚩 ML SaaS
– Jupyter fog workstation
– Remote GPU accelerator
|Research stage||👤 Product
|🚩 GPU cloud instances
– KVM support
|Research stage||👤 Product
As mentioned in Sonm goals definition, we find it extremely important to start gaining traction early on (early on the scale of fog computing platform development, which will take some years). Previously we expected multiple other projects and companies to start using Sonm PC sharing service, but something went wrong (will be discussed as a separate topic) and we decided to show a good example of how Sonm PC sharing was intended to be used.
The plan is to establish a commercial Software-as-a-Service heavily dependent on GPU computing, that can make use of huge benefit of Sonm PC sharing – GPU-based computing. Meet Rays – raysrender.com. Due to be launched in late July 2019.
We have big development plans for Rays and will keep working on it after its launch:
- Whitelabel feature – so other companies can use Rays under their own brand – Aug 2019.
- Support V-ray render engine – 2019.
- Support Redshift render engine – 2019.
- Develop plugins for Blender, Cinema 4D, Houdini, Maya.
- Mining farm conversion solution.
- Establish a crypto-friendly Rays clone. This version will accept SNM as payment.
We can either create 2-3 SaaS examples sharing our efforts between them or create 1 with focused development. Our alternative commercial SaaS ideas are in the research and design stage, meanwhile, we plan to focus App and Frontend teams on Rays. We will keep developing Rays until we support multiple render engines, multiple IDE integrations, multiple whitelabel installations and massive adoption.
Suppliers onboarding track
|✅ Elevated rewards program||Launched, active||👥 Business
|🏗 Upgrade to max consumer tier||In progress||👥 Business|
|🚩 Upgrade to SosnaTEE||Scheduled in Sep 2019|
|🚩 Upgrade to server tier (over 64 Gb RAM)|
Solving the chicken-and-egg problem, we work to attract hardware suppliers to join Sonm platform. We made a number of efforts and tried different strategies to find the one that fits us.
There are different possible types of hardware suppliers, that can be attracted to join Sonm platform. Without the comprehensive classification, that would be boring to read (we have such), we can say that the bigger the company is, the more famous it is – the higher the demand.
We made countless negotiations with mining farms, big and small possible suppliers from amateur to highly professional, and after all, we find it the most promising to work with people who have motivation for changes. The most important feature we look for in our suppliers is motivation:
- to change their common way of working with other mining software;
- to try our software, tolerate minor glitches and report bugs;
- willingness to upgrade their hardware according to our guideline.
We are proud to say that we found such motivated hardware suppliers and now we have over 100 Ethash Ghash/s computing power. This equals to more than 3000 GPUs of GeForce 1080Ti class. These people see joining Sonm as an opportunity and we are happy to provide a work scheme that is a replacement for traditional mining – PC sharing for commercial computing.
While there are a number of suppliers, who joined organically, that keep staying in Sonm for various reasons, the biggest onboarding happened when we introduced the Elevated Suppliers Program – our way of subsidizing hardware suppliers to stay in the system in anticipation of the Rays launch and commercial workloads.
Elevated Rewards Program
This is a community of private hardware owners, who receive elevated subsidized rewards on Sonm. Joining the Program was open for everyone in April, after a short time the Program was full. Today we do not accept new participants, but will open Program’s door once again after Rays launch and will keep accepting newcomers regularly in the future. However, there is currently more demand to join the Program, that the number of “seats” we can accept, so newcomers may be accepted on a lottery-basis.
It is still possible to join Sonm on a regular basis, as a marketplace, without Sonm subsidizing your deals. There is a minor municipal subsidizing open for everyone (with target profitability set above Ethereum mining profitability).
Upgrade to max consumer tier
Program is a two-way partnership. People in the Program:
- Receive truly higher revenues – $2.5 per day for a GeForce 1080Ti.
- Have obligations to keep machines online to stay in the program.
- Have the motivation to upgrade hardware – grow memory, CPU and eventually upgrade the motherboard to a server-grade.
We like the idea of growing together with our suppliers.
Upgrade to SosnaTEE
Once Trusted Platform is ready later this year, we plan to first release and test it on our suppliers from the Program for the reasons:
- Machines with Trusted Platforms will be subject to even higher rewards and we would like the people who support us by keeping machines in Sonm to have a chance to join first.
- Program participants are known and highly active in our tech chats, also they already have experience with Sonm.
- Program participants have some limited number of common PC configurations that we can learn, acquire testbeds and provide full testing as part of the release cycle.
Decentralization and tokenomics track
|✅ SNM to rent a PC||Done|
|🏗 Binance chain with gate masternodes||Work in progress
Approximate launch time: Aug-Sep 2019
(depends on acceptance and audit)
|👥 Blockchain dev team
|🚩 Suppliers deposits||Requirements specification
Approximate launch time: Sep 2019
|🚩 Masternodes||Requirements specification
Approximate launch time: Sep-Oct 2019 (depends on audit)
|🚩 Resource Allocation System market|
Existing Sonm platform features computing power as a commodity DEX. It operates using a utility ERC20 token – SNM. This is the basis we have now.
We see the future SNM token utility as the following:
- It is used as a means of value to pay for computing power – analog to compute-hour token (done).
- It is used as a license to operate as a supplier with Elevated Rewards (Suppliers deposits).
- It is required to operate a block producing masternode (see Masternodes).
- It is required to operate a gate masternode (see Masternodes and Binance chain).
- It is used as a license to operate as a client with bulk compute-time quotas (as part of the Resource Allocation System market).
Binance chain with gate masternodes
Many solutions already implemented in Sonm are ahead of many alternative projects. Now we decided to integrate with Binance chain:
- This will make SNM available on Binance DEX.
- We see great value in cooperating with a leading world crypto-world financial institution, which Binance is. Stronger the relationship – better for SNM token, more options for token holders.
This work is progress and we plan to cover it in a separate article.
Binance chain solution will have the first implementation of public gate masternodes. After Binance chain integration will be launched – gate masternodes will be activated, the community will be able to establish gate masternodes.
With the introduction of Elevated Rewards Program, which is to be followed up by increased computing demand from Rays, hardware suppliers are VIP passengers on this ship.
Indeed, working with Sonm:
- Provides an alternative to traditional mining
- with times higher rewards
- which are stable and predictable (you may rely on it and make business)
- that allows you to upgrade and gives a ticket to the future
On the other side, we would like SNM tokens to stay on the sidechain, which should help us reduce our utility token price volatility on exchanges. Keeping in mind that Sonm is (1) a free-to-use software, which is (2) developed at the cost of ICO funds, we find it reasonable for suppliers to have SNM deposit as a license to be granted access to highly profitable computing jobs.
It should be a normal situation that SNM utility token holder will:
- either use SNM to have his stake as a hardware supplier
- or use SNM to get computing power as a client
- or will sell his SNM to the one who will use it (as its mention in the above lines).
There is a comprehensive article about masternodes in Sonm, which is still up to date regarding masternode types and how they work: https://sonm.com/blog/sonm-masternodes-details-draft/
Gate masternodes are to be implemented as part of Binance chain integration project.
Block producing masternodes will be implemented after suppliers deposits. All prerequisites to establish them are met.
The existing Sonm PC sharing marketplace is a spot market. It means a client can rent what is available now, and a supplier can have a workload that is available now. If there is a disbalance in demand and supply, and this balance varies during the course of the day, there will always be someone unhappy.
Neither suppliers nor clients actually wish to operate on the spot market:
- Suppliers wish to have predictable, preferably guaranteed revenue. What will supplier do if there are no workloads at a given moment of time? The supplier will suffer.
- Clients need guaranteed quotas for compute hours. If a given client receives a peak load he needs to be able to get computing power. What if there are no free machines? The client will have damage to his business.
There must be a system on top of the spot market, that allows both suppliers and clients to have predictability and guarantees. We plan to implement such a system, which will be called the Resource Allocation System (RAS).
RAS will operate as a club (or pool) for suppliers and clients:
- Suppliers will have an entrance deposit (refundable with a delay), an evolution of plain suppliers’ deposit. Suppliers will be guaranteed to have workloads and will have a guaranteed revenue (currently this is implemented using Connor market making bots).
- The client will have both an entrance deposit and a bulk recurrent payment. Deposit size will determine this customer’s share in the RAS computing pool. The client will have guaranteed quota of compute hours, that will be always available (RAS suppliers will be obligated to switch to a RAS client if he has a valid quota). All clients will also share idle hours on a first-come-first-serve basis. They will also share expenses to cover suppliers idle time.
After all, a supplier will have a guaranteed revenue of X (per month or other time frames).
The client will have guaranteed compute hours Y (per month or other time frames) for a bulk price of Z.
Fog operating system
|🏗 Trusted platform||Work in progress
Approximate launch time: Sep 2019
|👥 Core dev team
|🚩 Monitoring and management||Approximate launch time: Oct 2019|
Currently, we have multiple Sonm install options for suppliers:
- SonmOS Linux distribution;
- Installation scripts;
- Manual install from Debian packages;
- Manual compile from source code;
- Prepacked virtual machine.
SonmOS was our first step in developing a Trusted Platform and a hardened Linux distribution. Currently, SonmOS is not encrypted and secured, but a future version – SosnaTEE – will be.
We are working to provide a complex security solution to make multi-tenancy fog computing, home PC and mining rig shared usage – secure and trusted. Our progress: we have a working prototype (works in the office) and a huge article draft about it (yet need to finish some details). This security solution will be launched this year.
The solution will include two major components: SosnaTEE and Kukarek:
- SosnaTEE is custom created hardened Linux distribution (evolution of SonmOS) that turns your home PC into a trusted compute node by cutting off your access to this machine runtime and performing remote machine attestation.
- Kukarek remote attestation service to work in pair with SosnaTEE OS. Kukarek performs remote OS checks to make sure the OS is not tempted with and that no local or remote access to machine runtime and data is possible.
The solution will transform the supplier’s PC into a black box, which doesn’t have any access, including root access, with paranoid encrypt-everything techniques.
Monitoring and management
Since SosnaTEE is unmanaged, it must provide adequate monitoring and management tools for hardware owner in a way, that supporting personnel can keep track of hardware metrics (mostly temperature sensors, fan speed, and disk utilization) and be able to schedule maintenance, while not having any access to the data on encrypted disks or program state in the memory.
Fog computing platform
|✅ PC sharing||Done|
|🚩 Resource Allocation System (RAS)||Research||👤 Product
|🚩 Fog compute and orchestration||Research||👤 Product
|🚩 Fog storage||Research||👤 Product
|🚩 Serverless fog|
The central idea about Sonm that puts everything together is fog computing platform. It can be seen from different angles as:
- distributed cloud;
- world data center.
This solution works as a sliced pie of technologies that work on top of each other:
- The bottom layer is the PC sharing platform. It allows individual suppliers and individual users to have a deal over some particular computer usage terms (equipment characteristics, time and price). At this level, we operate individual machines and their rental conditions.
- On top of the PC sharing platform, there is a Resource Allocation System (RAS), which allows compute time quotas and predictable resource provisioning. If we build a fog orchestrating service, it will require to have guaranteed quotas on PC compute time, otherwise, at peak hours there might not be any free machines to rent. On this level, we operate bulk computing time pools and quotas.
- A security solution is also required for the above layers to operate properly. Security solution must (1) reduce BFT requirements for the fog computing system itself and (2) protect the payload – the client’s compute jobs and their data.
- On top of RAS when we have some stable pool of computing power and given a security solution in place, fog compute and orchestration can be established. This should work similarly to cloud orchestrating software (kind of Kubernetes), but considering for limitations: (1) higher network latency, (2) lower network throughput, (3) storage may be slow and (4) trust works differently. Generally speaking, container orchestration can work in fog environment, but it requires deep modification.
- On top of the RAS and security solution, fog data storage can be arranged. There are existing storage solutions for both (1) trusted cloud environments (with fast networks) and (2) for untrusted distributed environment (with BFT requirement). First do not fit the fog, second are slow and expensive. Either novel solution is required or a deep modification of the existing one.
- As a final piece there may be a full-featured serverless fog solution on top of fog infrastructure (storage and compute/orchestration).
🚩 Libraries support and tool integration
🚩 CraftML (Sonm user) — 👥 External team
🚩 Rays whitelabel adopters — 👥 2 companies