Transaction broadcast succeeded but was not included on-chain

[jakamobiii]

Software Versions

OS : Linux
JVM : Oracle Corporation 1.8.0_161 amd64
Version : 4.8.0

Expected behaviour

After a transaction is broadcast successfully, it should be packaged into a block and included on-chain normally.

Actual behaviour

An abnormal behavior was observed in a private network: transactions that were broadcast successfully were not included on-chain.

• The private network consists of 1 SR node and 2 Fullnodes, all connected to each other.
• The two Fullnodes were started at the same time and immediately began synchronizing blocks from the SR (both were in the block-catching-up state).
• During the catch-up process, a transaction was broadcast to one of the Fullnodes:
  • The Fullnode returned a transaction broadcast success response.
  • However, the transaction never included on-chain, and it did not appear in any subsequent blocks.
• Based on network logs alone, the transaction seemed to be received and propagated by the nodes during the catch-up period, but it was not selected by the SR for block production.

Once block syncing completed, the issue disappeared and the system returned to normal behaviour.

Frequency

This issue had not occurred previously. It was discovered during a recent private network deployment and testing.

[zeusoo001]

@jakamobiii Based on your description, after the two fullnodes start up, they quickly complete block synchronization and enter broadcast state because their block heights are very close. Transactions can be broadcast between the two fullnodes, but since the fullnodes and the SR are in a synchronization state, transactions cannot be broadcast to the SR. Therefore, although the transaction appears to have been successfully broadcast, it is not actually recorded on the chain. It's best to wait until the fullnodes have completed synchronization before broadcasting the transaction.

[317787106]

@jakamobiii You should not broadcast any transaction through FullNode until it catched up the latest block.

[lxcmyf]

@jakamobiii
In the node synchronization phase, although the transaction is broadcast, it does not flow to SR, so it is not linked.

[jakamobiii]

Besides waiting for block synchronization to complete, are there other solutions?

[zeusoo001]

@jakamobiii It's also possible to broadcast through other nodes that have completed block synchronization.

[jakamobiii]

@zeusoo001 Another question is whether the API interface needs optimization. It shows a successful broadcast, but the transaction isn't actually recorded on the blockchain.

[zeusoo001]

@jakamobiii Yes, that's a very good question. I think it's necessary to optimize it. You can follow the progress of this issue.