DAWN-472 ⁃ Bandwidth Rate Limiting & Storage Usage Limits

DAWN-472 带宽速率限制和存储使用限制

bytemaster opened this issue on 6 Sep 2017

bytemaster在2017年9月6日开始了这个话题

第一篇内容链接点击此处，此篇是5个分篇中的第2篇

---

arhag commented on 7 Sep 2017

arhag在2017年9月7日回复了这个帖子

Dan and I had a discussion about this issue today. Here is the summary of my understanding of what we concluded (as well as some of my later additional thoughts regarding the database memory usage limit design that we discussed):

我和Dan今天就这个问题进行了谈论，以下是我对内容理解的总结（以及我后面加上了我们讨论中关于数据库内存使用限制设计的一些思考）

Bandwidth and Computation Rate Limiting

Transactions are received from the network and put inside a ring buffer to later be scheduled into cycles and processed by other threads. But prior to being placed in the ring buffer, the transaction must go through a filter (let us denote that filter the "bandwidth/computation filter”).

带宽和算力限制

从网络中接受到的交易，要把它放在待被其他线程处理的流程中，但放入之前这个交易必须要经过一次筛选（假设是被“带宽／算力过滤器”过滤了）

The bandwidth/computation filter maintains its own database (this is a bandwidth/computation quota database) separate from the databases holding contract state that are accessed during WebAssembly execution. The bandwidth/computation filter receives transactions from the network and then does a check / quota update for each of the accounts to be billed for bandwidth and computation usage of that transaction (these are the accounts that authorized the transaction).

带宽／算力过滤器维护自己的数据库（这是一个带宽／算力配额的数据库），分离那些持有合同并在WebAssembly期间被执行的数据库。带宽／算力过滤器接收每一个来自网络的交易，更新每一个账户的支付情况和配额，同时也为交易（这里指的是授权交易的账户）中的带宽和算力使用进行计费。

The check / quota update consists of:

1 accessing the current resource consumption values (calculated from the last update values and the current time) for each of the billed accounts;

2 adding to them a resource consumption delta that assumes the worst-case for each of the resource types (in this case only bandwidth and computation);

3 checking if any of those new calculated resource consumption values exceed the corresponding limit for each of the accounts;

4 and, assuming the checks passed, storing those new calculated resource consumption values in the database for each respective account.

检查／配额的更新包括：

1，访问每个付费用户的当前资源消耗值（从最新的更新值和当前时间计算）

2，增加一个假定每种资源都在最差的情况下（在这种情况下只有带宽和算力）的资源消耗增量

3，检查资源消耗值的新数据中是否有超过相应限额的账户

4，假定检查通过后，把这些资源消耗值的新数据存储在每一个账户中

If any of the limits are exceeded that transaction is rejected right there. Otherwise, it moves on to the next stage: placed in the ring buffer to later be scheduled and processed.

如果超过任何限制，交易将会被拒绝。与之相反的交易会进入到下一个阶段：待处理的流程中

To calculate a new resource consumption value assuming the worst-case, we need to determine what the worst-case values should actually be for both resource types in any given block. For the sake of easier implementation of the later resource consumption value updating stage, it would be helpful if these worst-case values were the same constants for all transactions. Ideally, they would remain constant over time as well, but it is still reasonable if they are only constant within any given block.

算出假定最坏情况下的资源消耗值，我们需要确定任意给出块中两种资源最差的值。如果这些最坏情况的值对于所有交易都是不变的，那对实现后面的资源消耗值更新阶段会很有帮助。理想情况下，它们不会因为时间的推移而变化，甚至能在任意给出的区块内保持不变。

The worst-case value to use for computation rate limiting is the maximum time that a contract is allowed to execute; so, for example, that might be 1 ms. The worst-case value to use for bandwidth rate limiting determines the maximum allowed size of a processed transaction. On one hand, one would wish that it not be too small, so that useful transactions are not rejected. On the other hand, larger worst-case values mean that the user needs to keep a larger buffer in their quotas to be able to transact at all, even if the transaction they broadcast will end up (even after being processed) remaining a very small size compared to the maximum allowed processed transaction size.

用于算力限制的最差值的情况是允许合同执行最长时间; 比如需要1毫秒。带宽速率限制的最差值情况决定了处理交易的最大范围值。一方面希望它允许的范围不会太小，以免正常交易被拒绝；另一方面，偏大的最差值意味着用户需要在其配额中保留一个足够大的空间才能保证交易，即使广播的交易已经结束（即使在处理之后）留下一个处理成功的小交易和最大值的交易进行比较。

During the later stages of the transactions lifetime, the transaction will ultimately either be rejected for a variety of possibly reasons or be accepted into a block. In either case, that transaction (which, if not rejected, is now a larger processed transaction which also includes metadata of how long the block producer claimed the contract took to run) needs to return back to the bandwidth/computation filter for the resource consumption value updating stage.

在交易的最后阶段，交易最终可能会因种种原因而被区块拒绝或被接受。在任何一种情况下，该交易（如果没有被拒绝，就到了被接受的大交易中，里面甚至还包括区块生产者声明花了多长时间运行合同的元数据）需要返回到带宽/计算过滤器的资源消耗值正在更新阶段当中。

This updating stage reduces the resource consumption values (for each of the relevant accounts) to reflect the fact that the transaction did not (at least in most cases) actually use the maximum amount of computational resources (e.g. contract execution took less than 1 ms of CPU time) or bandwidth resources (i.e the processed transaction size is less than the maximum processed transaction size). And in the case of transaction rejection, the values are updated by subtracting the worst-case delta values that were added earlier. This stage is in place to be fair to the users and not bill them at worst-case rates if they did not use up the resources to that level (or to not bill them at all if the transaction was rejected).

这个更新阶段减少的资源消耗值（针对每个相关账户）反映了每一次交易并没有（至少在大多数情况下）使用最大算力资源（例如，合同执行花费CPU少于1毫秒的时间）或带宽资源（即处理小于最高可处理量的交易）。在交易拒绝的情况下，通过减去先前添加的最差情况增量值来更新数据。这个阶段对于用户是公平的，要是在最差值的情况下使用资源（或者交易被拒绝），将不会被计费。

Database Memory Usage Limits

Limiting database memory usage works very differently compared to the bandwidth and computation rate limiting described above.

数据库内存使用限制

与上述的带宽和算力限制相比，数据库内存使用的限制非常不同。

The current memory usage of a contract would be stored in the regular database of the contract's account that is locked whenever access is needed to it during WebAssembly execution, rather than using a separate database as bandwidth and computation rate limiting do. EOS.IO database management code could then simply reject transactions that try to use more memory than they are allowed according to their current staked EOS tokens.

当WebAssembly执行期间需要访问它时，合约帐户的当前内存使用量将存储在合同帐户的常规数据库中，而不是像带宽和算力限制那样使用单独的数据库。EOS.IO数据库管理的代码可以轻易拒绝那些试图用更多的内存，而不是根据通过当前持有EOS代币来获得允许的交易。

The justification for this design comes from the following facts. First, the actual net change in a contract's database memory usage is not known until after the contract completes execution. Second, rejecting a transaction for using too much database memory requires at least two pieces of information: the amount of database memory the contract is using after contract execution and the number of EOS tokens staked in the contract account. The staked tokens information is stored in the regular database of the contract's account so that the eos contract can unstake an account's EOS tokens while only needing that in the scope. Therefore, determining whether to reject a transaction for using too much database memory would ultimately require a lock on the account of the contract(s) used in the transaction. So, perhaps it makes sense to just keep the memory usage information in the same place as the number of staked EOS tokens and require the lock on the contract account during WebAssembly execution.

这种设计的理由来自以下事实。首先，直到合同完成执行后，才知道其中数据库内存使用量的实际净变化。其次，拒绝使用过多数据库内存的交易需要至少两条信息：合同执行后合同使用的数据库内存量，以及合同帐户中持有的EOS代币数量。持有的代币信息存储在合同账户的常规数据库中，符合一定条件下，EOS合同可以取消账户中持有的EOS代币。因此，确定是否拒绝使用过多数据库内存的交易，最终会要求锁定交易中使用合约帐户的EOS代币。所以，将内存使用信息与持有大量EOS代币放在同一个地址，同时在WebAssembly执行期间要求锁定合同帐户时，可能才会有意义。

The negative consequence to that design (which I don't like but I haven't yet thought through other possible designs to see if they could actually have better performance than this design) is that anytime a contract needs to increase or decrease memory usage (even by a single byte) it needs to include the contract account as part of its required read/write scope.

这个设计的负面影响（我不喜欢它，但我还没找到其他比这个更好的设计）表现为，只要合约需要增加或减少内存使用量（即使是一个字节），它也需要作为合同中必不可少的一部分被包含在内，作出改变。

At least with some contracts, this isn't a big deal in terms of performance. A transfer transaction for a currency contract would only need to require the currency account as part of its scope if the recipient of the transfer did not already have a database entry for their balance (e.g. if this was the first time their account was interacting with that currency contract). But a transfer between two accounts that already had a currency balance could avoid requiring the currency account as part of the transaction's scope.

至少在某些合同中，这并不算什么大问题。如果转账的接收方还没有有余额的账号（例如，这是他们的账户第一次用这个代币进行转账），货币合约的转账交易只需要将currency账户作为其中的一部分。但互有余额的账户之间，转账不要求currency账户作为交易范围的一部分。

I do think this is a slight downgrade in terms of the user experience. While the required scopes could be automatically determined by running the user's transaction in a simulated node, this would not be a perfect process because it would be based on old blockchain state.

我认为这在用户体验方面评分不高。虽然所需的范围可以通过在模拟节点中运行用户的交易来自动确定。但它将基于旧的区块链状态，真不是一个完美的过程。

For example, Alice may run her transfer from her alice account to the bob account in the simulated node and determine that the only scopes required are {alice, bob} since the bob account already has a currency balance. But between broadcasting her transfer transaction and having it processed by a block producer, Bob may have sent his entire currency balance to someone else. If the currency contract was designed to free up database objects (for the sake of reclaiming memory) if an account's balance went to 0, then the bob account would not have a currency balance by the time Alice's transfer transaction was processed. Then when Alice's transfer transaction was actually being processed, it would be rejected because EOS.IO would determine that the memory usage of the currency contract needed to increase and yet the transaction had not declared currency as part of the scope.

例如，Alice可能会将她的Alice帐户转移到模拟节点中Bob的帐户，并确定{Alice, Bob}成立，唯一的要求是Bob帐户上有余额。但是，在广播她的转账交易并由一个区块打包的时候，Bob可能已经将他的全部余额转给别人了。如果货币合约设计为释放数据库对象（为了回收内存），账户的余额变为0，那在Alice转账交易完成后，Bob账户上将没有余额。到了Alice的转账交易实际上正在处理中的时候，它将会被拒绝。因为EOS.IO确定了合同的内存使用内容需要增加，但交易没有收到currency，这会成为其中一部分的信息。

Perhaps the currency contract designer will choose to not remove database objects even if an account's balance goes to 0. In this case the required scope determined by the simulated node may be over-conservative (requiring the currency when it was actually not necessary by the time the transaction was processed), but it would be far less likely to lead to the poor user-experience of having a broadcast transaction unexpectedly fail due to missing scopes. So, in that case the currency contract designer would trade better user-experience for increased memory usage (and therefore more EOS tokens locked).

也许货币合约设计者在账户的余额变为0的情况下，也不选择删除数据库对象，这时模拟节点需要确定的范围可能过于保守（要求currency在不是真的有必要的时候处理交易），但是由于范围缩小，让广播失败而引起差的用户体验反馈将会大大减少。因此，在这种情况下，货币合约设计者会迭代出更好的用户体验以增加内存使用量（因此锁定更多的EOS代币）。

But the point is that this design of database memory usage limits requires contract developers to make these sorts of trade-offs. Also, the previous example only looked at a currency contract. Other contracts may have more difficulty avoiding memory usage changes with most of their transactions. For example, it is highly unlikely that most transactions for an exchange contract will lead to no net change in memory usage (bid/asks need to be created and zero or more of existing bids/asks may need to be removed as orders are filled). In the case of an exchange contract, this design adds no additional overhead because all the transactions require the exchange account in the scope anyway (bids/asks are stored within the database of the exchange account). But there may be many other contracts we have not thought of that will be burdened with much less concurrency (and thus less transaction throughput) because of this design of database memory usage limits.

但重要的是，这种数据库内存使用限制的设计，要求合同开发者需要权衡这些问题。另外，前面的示例只能查看货币合约，其他合约避免可能在大部分交易中的内存使用变化时会遇到更多困难。例如，在交易合约中的大多数交易，都极不可能让内存使用情况没有变化（需要创建出价/询价，当订单被填满时，可能需要删除现有的出价/询价） 。在交易合同的情况下，这种设计不会增加额外的开销，因为所有的交易都需要交易账户在其数据库中（出价/询价将会被存储在交易账户的数据库中）。但是，由于这种数据库内存使用限制的设计，可能会有许多其他我们没有想到的合约会被更少的并发性(以及更少的交易吞吐量)所拖累。

（本文未完待续）

本文链接：https://github.com/EOSIO/eos/issues/353

翻译：Lochaiching

校正：Sheldon

本文首发于“EOS技术爱好者”微信公众号

这篇文章对你有用的话，可以用ERC-20钱包扫描以下地址给我们赞赏〜

二维码原文地址：0xBdE77CaFFf33970322c0F1f59F6B047de3AC88F9