diff --git a/README.md b/README.md index 0d0eeb6ef71..0f8b30704bf 100644 --- a/README.md +++ b/README.md @@ -1,49 +1,25 @@


-
- java-tron -

-

- Java implementation of the Tron Protocol + Java implementation of the TRON Protocol

- - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + +

## Table of Contents - [What’s TRON?](#whats-tron) - [Building the Source Code](#building-the-source-code) +- [Executables](#executables) - [Running java-tron](#running-java-tron) - [Community](#community) - [Contribution](#contribution) @@ -53,129 +29,171 @@ # What's TRON? -TRON is a project dedicated to building the infrastructure for a truly decentralized Internet. +TRON is building the foundational infrastructure for the decentralized internet ecosystem with a focus on high-performance, scalability, and security. + +- TRON Protocol: High-throughput(2000+ TPS), scalable blockchain OS (DPoS consensus) powering the TRON ecosystem. +- TRON Virtual Machine (TVM): EVM-compatible smart-contract engine for fast smart-contract execution. -- Tron Protocol, one of the largest blockchain-based operating systems in the world, offers scalable, high-availability and high-throughput support that underlies all the decentralized applications in the TRON ecosystem. +# Building the Source Code +Before building java-tron, make sure you have: +- Hardware with at least 4 CPU cores, 16 GB RAM, 10 GB free disk space for a smooth compilation process. +- Operating system: `Linux` or `macOS` (`Windows` is not supported). +- Git and correct JDK(version `8` or `17`) installed based on your CPU architecture. -- Tron Virtual Machine (TVM) allows anyone to develop decentralized applications (DAPPs) for themselves or their communities with smart contracts thereby making decentralized crowdfunding and token issuance easier than ever. +There are two ways to install the required dependencies: -TRON enables large-scale development and engagement. With over 2000 transactions per second (TPS), high concurrency, low latency, and massive data transmission. It is ideal for building decentralized entertainment applications. Free features and incentive systems allow developers to create premium app experiences for users. +- **Option 1: Automated script (recommended for quick setup)** -# Building the Source Code + Use the provided [`install_dependencies.sh`](install_dependencies.sh) script: -Building java-tron requires `git` package and 64-bit version of `Oracle JDK 1.8` to be installed, other JDK versions are not supported yet. Make sure you operate on `Linux` and `MacOS` operating systems. + ```bash + chmod +x install_dependencies.sh + ./install_dependencies.sh + ``` + > **Note**: For production-grade stability with JDK 8 on x86_64 architecture, Oracle JDK 8 is strongly recommended (the script installs OpenJDK 8). -Clone the repo and switch to the `master` branch +- **Option 2: Manual installation** + Follow the [Prerequisites and Installation Guide](https://tronprotocol.github.io/documentation-en/using_javatron/installing_javatron/#prerequisites-before-compiling-java-tron) for step-by-step instructions. + +Once all dependencies have been installed, download and compile java-tron by executing: ```bash -$ git clone https://github.com/tronprotocol/java-tron.git -$ cd java-tron -$ git checkout -t origin/master +git clone https://github.com/tronprotocol/java-tron.git +cd java-tron +git checkout -t origin/master +./gradlew clean build -x test ``` +* The parameter `-x test` indicates skipping the execution of test cases. +* If you encounter any error please refer to the [Compiling java-tron Source Code](https://tronprotocol.github.io/documentation-en/using_javatron/installing_javatron/#compiling-java-tron-source-code) documentation for troubleshooting steps. -then run the following command to build java-tron, the `FullNode.jar` file can be found in `java-tron/build/libs/` after build successfully. +# Executables -```bash -$ ./gradlew clean build -x test -``` +The java-tron project comes with several runnable artifacts and helper scripts found in the project root and build directories. -# Running java-tron +| Artifact/Script | Description | +| :---------------------- | :---------- | +| **`FullNode.jar`** | Main TRON node executable (generated in `build/libs/` after a successful build following the above guidance). Runs as a full node by default. `java -jar FullNode.jar --help` for command line options| +| **`Toolkit.jar`** | Node management utility (generated in `build/libs/`): partition, prune, copy, convert DBs; shadow-fork tool. [Usage](https://tronprotocol.github.io/documentation-en/using_javatron/toolkit/#toolkit-a-java-tron-node-maintenance-suite) | +| **`start.sh`** | Quick start script (x86_64, JDK 8) to download/build/run `FullNode.jar`. See the tool [guide](./shell.md). | +| **`start.sh.simple`** | Quick start script template (ARM64, JDK 17). See usage notes inside the script. | -Running java-tron requires 64-bit version of `Oracle JDK 1.8` to be installed, other JDK versions are not supported yet. Make sure you operate on `Linux` and `MacOS` operating systems. +# Running java-tron -Get the mainnet configuration file: [main_net_config.conf](https://github.com/tronprotocol/tron-deployment/blob/master/main_net_config.conf), other network configuration files can be found [here](https://github.com/tronprotocol/tron-deployment). +## Hardware Requirements for Mainnet -## Hardware Requirements +| Deployment Tier | CPU Cores | Memory | High-performance SSD Storage | Network Downstream | +|--------------------------|-------|--------|---------------------------|-----------------| +| FullNode (Minimum) | 8 | 16 GB | 200 GB ([Lite](https://tronprotocol.github.io/documentation-en/using_javatron/litefullnode/#lite-fullnode)) | ≥ 5 MBit/sec | +| FullNode (Stable) | 8 | 32 GB | 200 GB (Lite) 3.5 TB (Full) | ≥ 5 MBit/sec | +| FullNode (Recommend) | 16+ | 32 GB+ | 4 TB | ≥ 50 MBit/sec | +| Super Representative | 32+ | 64 GB+ | 4 TB | ≥ 50 MBit/sec | -Minimum: +> **Note**: For test networks, where transaction volume is significantly lower, you may operate with reduced hardware specifications. -- CPU with 8 cores -- 16GB RAM -- 3TB free storage space to sync the Mainnet +## Launching a full node -Recommended: +A full node acts as a gateway to the TRON network, exposing comprehensive interfaces via HTTP and RPC APIs. Through these endpoints, clients may execute asset transfers, deploy smart contracts, and invoke on-chain logic. It must join a TRON network to participate in the network's consensus and transaction processing. -- CPU with 16+ cores(32+ cores for a super representative) -- 32GB+ RAM(64GB+ for a super representative) -- High Performance SSD with at least 4TB free space -- 100+ MB/s download Internet service +### Network Types -## Running a full node for mainnet +The TRON network is mainly divided into: -Full node has full historical data, it is the entry point into the TRON network, it can be used by other processes as a gateway into the TRON network via HTTP and GRPC endpoints. You can interact with the TRON network through full node:transfer assets, deploy contracts, interact with contracts and so on. `-c` parameter specifies a configuration file to run a full node: +- **Main Network (Mainnet)** + The primary public blockchain where real value (TRX, TRC-20 tokens, etc.) is transacted, secured by a massive decentralized network. -```bash -$ nohup java -Xms9G -Xmx9G -XX:ReservedCodeCacheSize=256m \ - -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m \ - -XX:MaxDirectMemorySize=1G -XX:+PrintGCDetails \ - -XX:+PrintGCDateStamps -Xloggc:gc.log \ - -XX:+UseConcMarkSweepGC -XX:NewRatio=2 \ - -XX:+CMSScavengeBeforeRemark -XX:+ParallelRefProcEnabled \ - -XX:+HeapDumpOnOutOfMemoryError \ - -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 \ - -jar FullNode.jar -c main_net_config.conf >> start.log 2>&1 & -``` +- **[Nile Test Network (Testnet)](https://nileex.io/)** + A forward-looking testnet where new features and governance proposals are launched first for developers to experience. Consequently, its codebase is typically ahead of the Mainnet. -## Running a super representative node for mainnet +- **[Shasta Testnet](https://shasta.tronex.io/)** + Closely mirrors the Mainnet’s features and governance proposals. Its network parameters and software versions are kept in sync with the Mainnet, providing developers with a highly realistic environment for final testing. -Adding the `--witness` parameter to the startup command, full node will run as a super representative node. The super representative node supports all the functions of the full node and also supports block production. Before running, make sure you have a super representative account and get votes from others. Once the number of obtained votes ranks in the top 27, your super representative node will participate in block production. +- **Private Networks** + Customized TRON networks set up by private entities for testing, development, or specific use cases. -Fill in the private key of a super representative address into the `localwitness` list in the `main_net_config.conf`. Here is an example: +Network selection is performed by specifying the appropriate configuration file upon full-node startup. Mainnet configuration: [config.conf](framework/src/main/resources/config.conf); Nile testnet configuration: [config-nile.conf](https://github.com/tron-nile-testnet/nile-testnet/blob/master/framework/src/main/resources/config-nile.conf) +### 1. Join the TRON main network +Launch a main-network full node with the built-in default configuration: +```bash +java -jar ./build/libs/FullNode.jar ``` - localwitness = [ - - ] + +> For production deployments or long-running Mainnet nodes, please refer to the [JVM Parameter Optimization for FullNode](https://tronprotocol.github.io/documentation-en/using_javatron/installing_javatron/#jvm-parameter-optimization-for-mainnet-fullnode-deployment) guide for the recommended Java command configuration. + +Using the below command, you can monitor the blocks syncing progress: +```bash +tail -f ./logs/tron.log ``` -then run the following command to start the node: +Use [TronScan](https://tronscan.org/#/), TRON's official block explorer, to view main network transactions, blocks, accounts, witness voting, and governance metrics, etc. + +### 2. Join Nile test network +Utilize the `-c` flag to direct the node to the configuration file corresponding to the desired network. Since Nile TestNet may incorporate features not yet available on the MainNet, it is **strongly advised** to compile the source code following the [Building the Source Code](https://github.com/tron-nile-testnet/nile-testnet/blob/master/README.md#building-the-source-code) instructions for the Nile TestNet. ```bash -$ nohup java -Xms9G -Xmx9G -XX:ReservedCodeCacheSize=256m \ - -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m \ - -XX:MaxDirectMemorySize=1G -XX:+PrintGCDetails \ - -XX:+PrintGCDateStamps -Xloggc:gc.log \ - -XX:+UseConcMarkSweepGC -XX:NewRatio=2 \ - -XX:+CMSScavengeBeforeRemark -XX:+ParallelRefProcEnabled \ - -XX:+HeapDumpOnOutOfMemoryError \ - -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 \ - -jar FullNode.jar --witness -c main_net_config.conf >> start.log 2>&1 & +java -jar ./build/libs/FullNode.jar -c config-nile.conf ``` -## Quick Start Tool +Nile resources: explorer, faucet, wallet, developer docs, and network statistics at [nileex.io](https://nileex.io/). -An easier way to build and run java-tron is to use `start.sh`. `start.sh` is a quick start script written in the Shell language. You can use it to build and run java-tron quickly and easily. +### 3. Access Shasta test network +Shasta does not accept public node peers. Programmatic access is available via TronGrid endpoints; see [TronGrid Service](https://developers.tron.network/docs/trongrid) for details. -Here are some common use cases of the scripting tool +Shasta resources: explorer, faucet, wallet, developer docs, and network statistics at [shastaex.io](https://shasta.tronex.io/). -- Use `start.sh` to start a full node with the downloaded `FullNode.jar` -- Use `start.sh` to download the latest `FullNode.jar` and start a full node. -- Use `start.sh` to download the latest source code and compile a `FullNode.jar` and then start a full node. +### 4. Set up a private network +To set up a private network for testing or development, follow the [Private Network guidance](https://tronprotocol.github.io/documentation-en/using_javatron/private_network/). -For more details, please refer to the tool [guide](./shell.md). +## Running a super representative node -## Run inside Docker container +To operate the node as a Super Representative (SR), append the `--witness` parameter to the standard launch command. An SR node inherits every capability of a FullNode and additionally participates in block production. Refer to the [Super Representative documentation](https://tronprotocol.github.io/documentation-en/mechanism-algorithm/sr/) for eligibility requirements. -One of the quickest ways to get `java-tron` up and running on your machine is by using Docker: +Fill in the private key of your SR account into the `localwitness` list in the configuration file. Here is an example: -```shell -$ docker run -d --name="java-tron" \ - -v /your_path/output-directory:/java-tron/output-directory \ - -v /your_path/logs:/java-tron/logs \ - -p 8090:8090 -p 18888:18888 -p 50051:50051 \ - tronprotocol/java-tron \ - -c /java-tron/config/main_net_config.conf ``` + localwitness = [ + + ] +``` +Check [Starting a Block Production Node](https://tronprotocol.github.io/documentation-en/using_javatron/installing_javatron/#starting-a-block-production-node) for more details. +You could also test the process by connecting to a testnet or setting up a private network. -This will mount the `output-directory` and `logs` directories on the host, the docker.sh tool can also be used to simplify the use of docker, see more [here](docker/docker.md). +## Programmatically interfacing FullNode -# Community +Upon the FullNode startup successfully, interaction with the TRON network is facilitated through a comprehensive suite of programmatic interfaces exposed by java-tron: +- **HTTP API**: See the complete [HTTP API reference and endpoint list](https://tronprotocol.github.io/documentation-en/api/http/). +- **gRPC**: High-performance APIs suitable for service-to-service integration. See the supported [gRPC reference](https://tronprotocol.github.io/documentation-en/api/rpc/). +- **JSON-RPC**: Provides Ethereum-compatible JSON-RPC methods for logs, transactions and contract calls, etc. See the supported [JSON-RPC methods](https://tronprotocol.github.io/documentation-en/api/json-rpc/). -[Tron Developers & SRs](https://discord.gg/hqKvyAM) is Tron's official Discord channel. Feel free to join this channel if you have any questions. +Enable or disable each interface in the configuration file: + +``` +node { + http { + fullNodeEnable = true + fullNodePort = 8090 + } + + jsonrpc { + httpFullNodeEnable = true + httpFullNodePort = 8545 + } + + rpc { + enable = true + port = 9090 + } +} +``` +When exposing any of these APIs to a public interface, ensure the node is protected with appropriate authentication, rate limiting, and network access controls in line with your security requirements. + +Public hosted HTTP endpoints for both mainnet and testnet are provided by TronGrid. Please refer to the [TRON Network HTTP Endpoints](https://developers.tron.network/docs/connect-to-the-tron-network#tron-network-http-endpoints) for the latest list. For supported methods and request formats, see the HTTP API reference above. + +# Community -[Core Devs Community](https://t.me/troncoredevscommunity) is the Telegram channel for java-tron community developers. If you want to contribute to java-tron, please join this channel. +[TRON Developers & SRs](https://discord.gg/hqKvyAM) is TRON's official Discord channel. Feel free to join this channel if you have any questions. -[tronprotocol/allcoredev](https://gitter.im/tronprotocol/allcoredev) is the official Gitter channel for developers. +The [Core Devs Community](https://t.me/troncoredevscommunity) and [TRON Official Developer Group](https://t.me/TronOfficialDevelopersGroupEn) are Telegram channels specifically designed for java-tron community developers to engage in technical discussions. # Contribution @@ -184,9 +202,10 @@ Thank you for considering to help out with the source code! If you'd like to con # Resources - [Medium](https://medium.com/@coredevs) java-tron's official technical articles are published there. -- [Documentation](https://tronprotocol.github.io/documentation-en/introduction/) java-tron's official technical documentation website. -- [Test network](http://nileex.io/) A stable test network of TRON contributed by TRON community. -- [Tronscan](https://tronscan.org/#/) TRON network blockchain browser. +- [Documentation](https://tronprotocol.github.io/documentation-en/) and [TRON Developer Hub](https://developers.tron.network/) serve as java-tron’s primary documentation websites. +- [TronScan](https://tronscan.org/#/) TRON main network blockchain browser. +- [Nile Test network](http://nileex.io/) A stable test network of TRON contributed by TRON community. +- [Shasta Test network](https://shasta.tronex.io/) A stable test network of TRON contributed by TRON community. - [Wallet-cli](https://github.com/tronprotocol/wallet-cli) TRON network wallet using command line. - [TIP](https://github.com/tronprotocol/tips) TRON Improvement Proposal (TIP) describes standards for the TRON network. - [TP](https://github.com/tronprotocol/tips/tree/master/tp) TRON Protocol (TP) describes standards already implemented in TRON network but not published as a TIP. diff --git a/actuator/src/main/java/org/tron/core/actuator/AssetIssueActuator.java b/actuator/src/main/java/org/tron/core/actuator/AssetIssueActuator.java index 331b45f106a..618a9fb191e 100644 --- a/actuator/src/main/java/org/tron/core/actuator/AssetIssueActuator.java +++ b/actuator/src/main/java/org/tron/core/actuator/AssetIssueActuator.java @@ -24,10 +24,12 @@ import java.util.List; import java.util.Objects; import lombok.extern.slf4j.Slf4j; +import org.tron.common.math.StrictMathWrapper; import org.tron.common.utils.DecodeUtil; import org.tron.core.capsule.AccountCapsule; import org.tron.core.capsule.AssetIssueCapsule; import org.tron.core.capsule.TransactionResultCapsule; +import org.tron.core.config.Parameter.ForkBlockVersionEnum; import org.tron.core.exception.BalanceInsufficientException; import org.tron.core.exception.ContractExeException; import org.tron.core.exception.ContractValidateException; @@ -263,6 +265,16 @@ public boolean validate() throws ContractValidateException { "frozenDuration must be less than " + maxFrozenSupplyTime + " days " + "and more than " + minFrozenSupplyTime + " days"); } + // make sure FrozenSupply.expireTime not overflow + if (chainBaseManager.getForkController().pass(ForkBlockVersionEnum.VERSION_4_8_1)) { + long frozenPeriod = next.getFrozenDays() * FROZEN_PERIOD; + try { + StrictMathWrapper.addExact(assetIssueContract.getStartTime(), frozenPeriod); + } catch (ArithmeticException e) { + throw new ContractValidateException( + "Start time and frozen days would cause expire time overflow"); + } + } remainSupply -= next.getFrozenAmount(); } diff --git a/actuator/src/main/java/org/tron/core/actuator/ProposalCreateActuator.java b/actuator/src/main/java/org/tron/core/actuator/ProposalCreateActuator.java index a3639cca07f..e0044c4958d 100755 --- a/actuator/src/main/java/org/tron/core/actuator/ProposalCreateActuator.java +++ b/actuator/src/main/java/org/tron/core/actuator/ProposalCreateActuator.java @@ -9,7 +9,6 @@ import java.util.Map; import java.util.Objects; import lombok.extern.slf4j.Slf4j; -import org.tron.common.parameter.CommonParameter; import org.tron.common.utils.DecodeUtil; import org.tron.common.utils.StringUtil; import org.tron.core.capsule.ProposalCapsule; @@ -53,7 +52,7 @@ public boolean execute(Object result) throws ContractExeException { long currentMaintenanceTime = chainBaseManager.getDynamicPropertiesStore().getNextMaintenanceTime(); - long now3 = now + CommonParameter.getInstance().getProposalExpireTime(); + long now3 = now + chainBaseManager.getDynamicPropertiesStore().getProposalExpireTime(); long round = (now3 - currentMaintenanceTime) / maintenanceTimeInterval; long expirationTime = currentMaintenanceTime + (round + 1) * maintenanceTimeInterval; diff --git a/actuator/src/main/java/org/tron/core/actuator/ShieldedTransferActuator.java b/actuator/src/main/java/org/tron/core/actuator/ShieldedTransferActuator.java index 2773dc07d26..338e948f304 100644 --- a/actuator/src/main/java/org/tron/core/actuator/ShieldedTransferActuator.java +++ b/actuator/src/main/java/org/tron/core/actuator/ShieldedTransferActuator.java @@ -170,28 +170,27 @@ private void executeShielded(List spends, List= MAX_PROPOSAL_EXPIRE_TIME) { + throw new ContractValidateException( + "This value[PROPOSAL_EXPIRE_TIME] is only allowed to be greater than " + + MIN_PROPOSAL_EXPIRE_TIME + " and less than " + + MAX_PROPOSAL_EXPIRE_TIME + "!"); + } + break; + } default: break; } @@ -921,7 +953,9 @@ public enum ProposalType { // current value, value range ALLOW_TVM_CANCUN(83), // 0, 1 ALLOW_STRICT_MATH(87), // 0, 1 CONSENSUS_LOGIC_OPTIMIZATION(88), // 0, 1 - ALLOW_TVM_BLOB(89); // 0, 1 + ALLOW_TVM_BLOB(89), // 0, 1 + PROPOSAL_EXPIRE_TIME(92), // (0, 31536003000) + ALLOW_TVM_SELFDESTRUCT_RESTRICTION(94); // 0, 1 private long code; diff --git a/actuator/src/main/java/org/tron/core/vm/EnergyCost.java b/actuator/src/main/java/org/tron/core/vm/EnergyCost.java index b758438c940..d47f716943f 100644 --- a/actuator/src/main/java/org/tron/core/vm/EnergyCost.java +++ b/actuator/src/main/java/org/tron/core/vm/EnergyCost.java @@ -55,6 +55,7 @@ public class EnergyCost { private static final long EXT_CODE_SIZE = 20; private static final long EXT_CODE_HASH = 400; private static final long SUICIDE = 0; + private static final long SUICIDE_V2 = 5000; private static final long STOP = 0; private static final long CREATE_DATA = 200; private static final long TLOAD = 100; @@ -289,6 +290,14 @@ public static long getSuicideCost2(Program program) { return getSuicideCost(program); } + public static long getSuicideCost3(Program program) { + DataWord inheritorAddress = program.getStack().peek(); + if (isDeadAccount(program, inheritorAddress)) { + return SUICIDE_V2 + NEW_ACCT_CALL; + } + return SUICIDE_V2; + } + public static long getBalanceCost(Program ignored) { return BALANCE; } diff --git a/actuator/src/main/java/org/tron/core/vm/OperationActions.java b/actuator/src/main/java/org/tron/core/vm/OperationActions.java index f10fb37dd7e..0d978743a5e 100644 --- a/actuator/src/main/java/org/tron/core/vm/OperationActions.java +++ b/actuator/src/main/java/org/tron/core/vm/OperationActions.java @@ -1072,4 +1072,19 @@ public static void suicideAction(Program program) { program.stop(); } + public static void suicideAction2(Program program) { + if (program.isStaticCall()) { + throw new Program.StaticCallModificationException(); + } + + if (!program.canSuicide2()) { + program.getResult().setRevert(); + } else { + DataWord address = program.stackPop(); + program.suicide2(address); + } + + program.stop(); + } + } diff --git a/actuator/src/main/java/org/tron/core/vm/OperationRegistry.java b/actuator/src/main/java/org/tron/core/vm/OperationRegistry.java index be29238a775..f6140107efb 100644 --- a/actuator/src/main/java/org/tron/core/vm/OperationRegistry.java +++ b/actuator/src/main/java/org/tron/core/vm/OperationRegistry.java @@ -79,6 +79,10 @@ public static JumpTable getTable() { adjustForFairEnergy(table); } + if (VMConfig.allowTvmSelfdestructRestriction()) { + adjustSelfdestruct(table); + } + return table; } @@ -695,4 +699,11 @@ public static void appendCancunOperations(JumpTable table) { OperationActions::blobBaseFeeAction, tvmBlobProposal)); } + + public static void adjustSelfdestruct(JumpTable table) { + table.set(new Operation( + Op.SUICIDE, 1, 0, + EnergyCost::getSuicideCost3, + OperationActions::suicideAction2)); + } } diff --git a/actuator/src/main/java/org/tron/core/vm/PrecompiledContracts.java b/actuator/src/main/java/org/tron/core/vm/PrecompiledContracts.java index 7913d7928fa..654a76db33b 100644 --- a/actuator/src/main/java/org/tron/core/vm/PrecompiledContracts.java +++ b/actuator/src/main/java/org/tron/core/vm/PrecompiledContracts.java @@ -16,6 +16,7 @@ import static org.tron.common.utils.ByteUtil.parseWord; import static org.tron.common.utils.ByteUtil.stripLeadingZeroes; import static org.tron.core.config.Parameter.ChainConstant.TRX_PRECISION; +import static org.tron.core.vm.VMConstant.SIG_LENGTH; import com.google.protobuf.ByteString; @@ -41,6 +42,7 @@ import org.apache.commons.lang3.tuple.Triple; import org.tron.common.crypto.Blake2bfMessageDigest; import org.tron.common.crypto.Hash; +import org.tron.common.crypto.Rsv; import org.tron.common.crypto.SignUtils; import org.tron.common.crypto.SignatureInterface; import org.tron.common.crypto.zksnark.BN128; @@ -201,7 +203,7 @@ public class PrecompiledContracts { public static PrecompiledContract getOptimizedContractForConstant(PrecompiledContract contract) { try { Constructor constructor = contract.getClass().getDeclaredConstructor(); - return (PrecompiledContracts.PrecompiledContract) constructor.newInstance(); + return (PrecompiledContracts.PrecompiledContract) constructor.newInstance(); } catch (Exception e) { throw new RuntimeException(e); } @@ -352,22 +354,13 @@ private static byte[] encodeMultiRes(byte[]... words) { } private static byte[] recoverAddrBySign(byte[] sign, byte[] hash) { - byte v; - byte[] r; - byte[] s; byte[] out = null; if (ArrayUtils.isEmpty(sign) || sign.length < 65) { return new byte[0]; } try { - r = Arrays.copyOfRange(sign, 0, 32); - s = Arrays.copyOfRange(sign, 32, 64); - v = sign[64]; - if (v < 27) { - v += 27; - } - - SignatureInterface signature = SignUtils.fromComponents(r, s, v, + Rsv rsv = Rsv.fromSignature(sign); + SignatureInterface signature = SignUtils.fromComponents(rsv.getR(), rsv.getS(), rsv.getV(), CommonParameter.getInstance().isECKeyCryptoEngine()); if (signature.validateComponents()) { out = SignUtils.signatureToAddress(hash, signature, @@ -403,6 +396,20 @@ private static byte[][] extractBytesArray(DataWord[] words, int offset, byte[] d return bytesArray; } + private static byte[][] extractSigArray(DataWord[] words, int offset, byte[] data) { + if (offset > words.length - 1) { + return new byte[0][]; + } + int len = words[offset].intValueSafe(); + byte[][] bytesArray = new byte[len][]; + for (int i = 0; i < len; i++) { + int bytesOffset = words[offset + i + 1].intValueSafe() / WORD_SIZE; + bytesArray[i] = extractBytes(data, (bytesOffset + offset + 2) * WORD_SIZE, + SIG_LENGTH); + } + return bytesArray; + } + private static byte[] extractBytes(byte[] data, int offset, int len) { return Arrays.copyOfRange(data, offset, offset + len); } @@ -944,8 +951,15 @@ public Pair execute(byte[] rawData) { byte[] hash = Sha256Hash.hash(CommonParameter .getInstance().isECKeyCryptoEngine(), combine); - byte[][] signatures = extractBytesArray( - words, words[3].intValueSafe() / WORD_SIZE, rawData); + if (VMConfig.allowTvmSelfdestructRestriction()) { + int sigArraySize = words[words[3].intValueSafe() / WORD_SIZE].intValueSafe(); + if (sigArraySize > MAX_SIZE) { + return Pair.of(true, DATA_FALSE); + } + } + byte[][] signatures = VMConfig.allowTvmSelfdestructRestriction() ? + extractSigArray(words, words[3].intValueSafe() / WORD_SIZE, rawData) : + extractBytesArray(words, words[3].intValueSafe() / WORD_SIZE, rawData); if (signatures.length == 0 || signatures.length > MAX_SIZE) { return Pair.of(true, DATA_FALSE); @@ -1029,8 +1043,18 @@ private Pair doExecute(byte[] data) throws InterruptedException, ExecutionException { DataWord[] words = DataWord.parseArray(data); byte[] hash = words[0].getData(); - byte[][] signatures = extractBytesArray( - words, words[1].intValueSafe() / WORD_SIZE, data); + + if (VMConfig.allowTvmSelfdestructRestriction()) { + int sigArraySize = words[words[1].intValueSafe() / WORD_SIZE].intValueSafe(); + int addrArraySize = words[words[2].intValueSafe() / WORD_SIZE].intValueSafe(); + if (sigArraySize > MAX_SIZE || addrArraySize > MAX_SIZE) { + return Pair.of(true, DATA_FALSE); + } + } + + byte[][] signatures = VMConfig.allowTvmSelfdestructRestriction() ? + extractSigArray(words, words[1].intValueSafe() / WORD_SIZE, data) : + extractBytesArray(words, words[1].intValueSafe() / WORD_SIZE, data); byte[][] addresses = extractBytes32Array( words, words[2].intValueSafe() / WORD_SIZE); int cnt = signatures.length; diff --git a/actuator/src/main/java/org/tron/core/vm/VM.java b/actuator/src/main/java/org/tron/core/vm/VM.java index 2150df04c64..b1d7b027601 100644 --- a/actuator/src/main/java/org/tron/core/vm/VM.java +++ b/actuator/src/main/java/org/tron/core/vm/VM.java @@ -108,7 +108,10 @@ public static void play(Program program, JumpTable jumpTable) { } catch (JVMStackOverFlowException | OutOfTimeException e) { throw e; } catch (RuntimeException e) { - if (StringUtils.isEmpty(e.getMessage())) { + // https://openjdk.org/jeps/358 + // https://bugs.openjdk.org/browse/JDK-8220715 + // since jdk 14, the NullPointerExceptions message is not empty + if (e instanceof NullPointerException || StringUtils.isEmpty(e.getMessage())) { logger.warn("Unknown Exception occurred, tx id: {}", Hex.toHexString(program.getRootTransactionId()), e); program.setRuntimeFailure(new RuntimeException("Unknown Exception")); diff --git a/actuator/src/main/java/org/tron/core/vm/VMConstant.java b/actuator/src/main/java/org/tron/core/vm/VMConstant.java index abbb6ae6d38..266224a1502 100644 --- a/actuator/src/main/java/org/tron/core/vm/VMConstant.java +++ b/actuator/src/main/java/org/tron/core/vm/VMConstant.java @@ -4,6 +4,7 @@ public class VMConstant { public static final int CONTRACT_NAME_LENGTH = 32; public static final int MIN_TOKEN_ID = 1_000_000; + public static final int SIG_LENGTH = 65; // Numbers public static final int ONE_HUNDRED = 100; diff --git a/actuator/src/main/java/org/tron/core/vm/config/ConfigLoader.java b/actuator/src/main/java/org/tron/core/vm/config/ConfigLoader.java index 1ec8a75d669..e0ebca329cd 100644 --- a/actuator/src/main/java/org/tron/core/vm/config/ConfigLoader.java +++ b/actuator/src/main/java/org/tron/core/vm/config/ConfigLoader.java @@ -44,6 +44,7 @@ public static void load(StoreFactory storeFactory) { VMConfig.initAllowTvmCancun(ds.getAllowTvmCancun()); VMConfig.initDisableJavaLangMath(ds.getConsensusLogicOptimization()); VMConfig.initAllowTvmBlob(ds.getAllowTvmBlob()); + VMConfig.initAllowTvmSelfdestructRestriction(ds.getAllowTvmSelfdestructRestriction()); } } } diff --git a/actuator/src/main/java/org/tron/core/vm/program/ContractState.java b/actuator/src/main/java/org/tron/core/vm/program/ContractState.java index e095201e393..c6347b9a072 100644 --- a/actuator/src/main/java/org/tron/core/vm/program/ContractState.java +++ b/actuator/src/main/java/org/tron/core/vm/program/ContractState.java @@ -121,6 +121,16 @@ public void updateContractState(byte[] address, ContractStateCapsule contractSta repository.updateContractState(address, contractStateCapsule); } + @Override + public void putNewContract(byte[] address) { + repository.putNewContract(address); + } + + @Override + public boolean isNewContract(byte[] address) { + return repository.isNewContract(address); + } + @Override public void updateAccount(byte[] address, AccountCapsule accountCapsule) { repository.updateAccount(address, accountCapsule); diff --git a/actuator/src/main/java/org/tron/core/vm/program/Memory.java b/actuator/src/main/java/org/tron/core/vm/program/Memory.java index 4249a7e2634..2c43c02e138 100644 --- a/actuator/src/main/java/org/tron/core/vm/program/Memory.java +++ b/actuator/src/main/java/org/tron/core/vm/program/Memory.java @@ -98,7 +98,7 @@ public void write(int address, byte[] data, int dataSize, boolean limited) { public void extendAndWrite(int address, int allocSize, byte[] data) { extend(address, allocSize); - write(address, data, data.length, false); + write(address, data, allocSize, false); } public void extend(int address, int size) { diff --git a/actuator/src/main/java/org/tron/core/vm/program/Program.java b/actuator/src/main/java/org/tron/core/vm/program/Program.java index 5da0b02ecb7..80d972041dc 100644 --- a/actuator/src/main/java/org/tron/core/vm/program/Program.java +++ b/actuator/src/main/java/org/tron/core/vm/program/Program.java @@ -515,6 +515,72 @@ public void suicide(DataWord obtainerAddress) { getResult().addDeleteAccount(this.getContractAddress()); } + public void suicide2(DataWord obtainerAddress) { + + byte[] owner = getContextAddress(); + boolean isNewContract = getContractState().isNewContract(owner); + if (isNewContract) { + suicide(obtainerAddress); + return; + } + + byte[] obtainer = obtainerAddress.toTronAddress(); + + long balance = getContractState().getBalance(owner); + + if (logger.isDebugEnabled()) { + logger.debug("Transfer to: [{}] heritage: [{}]", + Hex.toHexString(obtainer), + balance); + } + + increaseNonce(); + + InternalTransaction internalTx = addInternalTx(null, owner, obtainer, balance, null, + "suicide", nonce, getContractState().getAccount(owner).getAssetMapV2()); + + if (FastByteComparisons.isEqual(owner, obtainer)) { + return; + } + + if (VMConfig.allowTvmVote()) { + withdrawRewardAndCancelVote(owner, getContractState()); + balance = getContractState().getBalance(owner); + if (internalTx != null && balance != internalTx.getValue()) { + internalTx.setValue(balance); + } + } + + // transfer balance and trc10 + createAccountIfNotExist(getContractState(), obtainer); + try { + MUtil.transfer(getContractState(), owner, obtainer, balance); + if (VMConfig.allowTvmTransferTrc10()) { + MUtil.transferAllToken(getContractState(), owner, obtainer); + } + } catch (ContractValidateException e) { + if (VMConfig.allowTvmConstantinople()) { + throw new TransferException( + "transfer all token or transfer all trx failed in suicide: %s", e.getMessage()); + } + throw new BytecodeExecutionException("transfer failure"); + } + + // transfer freeze + if (VMConfig.allowTvmFreeze()) { + transferDelegatedResourceToInheritor(owner, obtainer, getContractState()); + } + + // transfer freezeV2 + if (VMConfig.allowTvmFreezeV2()) { + long expireUnfrozenBalance = + transferFrozenV2BalanceToInheritor(owner, obtainer, getContractState()); + if (expireUnfrozenBalance > 0 && internalTx != null) { + internalTx.setValue(internalTx.getValue() + expireUnfrozenBalance); + } + } + } + public Repository getContractState() { return this.contractState; } @@ -544,6 +610,11 @@ private void transferDelegatedResourceToInheritor(byte[] ownerAddr, byte[] inher // transfer all kinds of frozen balance to BlackHole repo.addBalance(inheritorAddr, frozenBalanceForBandwidthOfOwner + frozenBalanceForEnergyOfOwner); + + if (VMConfig.allowTvmSelfdestructRestriction()) { + clearOwnerFreeze(ownerCapsule); + repo.updateAccount(ownerAddr, ownerCapsule); + } } private long transferFrozenV2BalanceToInheritor(byte[] ownerAddr, byte[] inheritorAddr, Repository repo) { @@ -609,6 +680,11 @@ private long transferFrozenV2BalanceToInheritor(byte[] ownerAddr, byte[] inherit return expireUnfrozenBalance; } + private void clearOwnerFreeze(AccountCapsule ownerCapsule) { + ownerCapsule.setFrozenForBandwidth(0, 0); + ownerCapsule.setFrozenForEnergy(0, 0); + } + private void clearOwnerFreezeV2(AccountCapsule ownerCapsule) { ownerCapsule.clearFrozenV2(); ownerCapsule.setNetUsage(0); @@ -666,6 +742,38 @@ public boolean canSuicide() { // return freezeCheck && voteCheck; } + public boolean canSuicide2() { + byte[] owner = getContextAddress(); + AccountCapsule accountCapsule = getContractState().getAccount(owner); + + return freezeV1Check(accountCapsule) && freezeV2Check(accountCapsule); + } + + private boolean freezeV1Check(AccountCapsule accountCapsule) { + if (!VMConfig.allowTvmFreeze()) { + return true; + } + + // check freeze + long now = getContractState().getDynamicPropertiesStore().getLatestBlockHeaderTimestamp(); + // bandwidth + if (accountCapsule.getFrozenCount() > 0 + && accountCapsule.getFrozenList().stream() + .anyMatch(frozen -> frozen.getExpireTime() > now)) { + return false; + } + // energy + Protocol.Account.Frozen frozenEnergy = + accountCapsule.getAccountResource().getFrozenBalanceForEnergy(); + if (frozenEnergy.getFrozenBalance() > 0 && frozenEnergy.getExpireTime() > now) { + return false; + } + + // check delegate + return accountCapsule.getDelegatedFrozenBalanceForBandwidth() == 0 + && accountCapsule.getDelegatedFrozenBalanceForEnergy() == 0; + } + private boolean freezeV2Check(AccountCapsule accountCapsule) { if (!VMConfig.allowTvmFreezeV2()) { return true; @@ -1638,7 +1746,11 @@ public void callToPrecompiledAddress(MessageCall msg, } } - this.memorySave(msg.getOutDataOffs().intValue(), out.getRight()); + if (VMConfig.allowTvmSelfdestructRestriction()) { + this.memorySave(msg.getOutDataOffs().intValueSafe(), msg.getOutDataSize().intValueSafe(), out.getRight()); + } else { + this.memorySave(msg.getOutDataOffs().intValue(), out.getRight()); + } } } diff --git a/actuator/src/main/java/org/tron/core/vm/program/invoke/ProgramInvokeImpl.java b/actuator/src/main/java/org/tron/core/vm/program/invoke/ProgramInvokeImpl.java index 7997aaedcd5..ede20103609 100644 --- a/actuator/src/main/java/org/tron/core/vm/program/invoke/ProgramInvokeImpl.java +++ b/actuator/src/main/java/org/tron/core/vm/program/invoke/ProgramInvokeImpl.java @@ -314,7 +314,7 @@ public boolean equals(Object o) { @Override public int hashCode() { - return new Integer(Boolean.valueOf(byTestingSuite).hashCode() + return Boolean.valueOf(byTestingSuite).hashCode() + Boolean.valueOf(byTransaction).hashCode() + address.hashCode() + balance.hashCode() @@ -326,8 +326,7 @@ public int hashCode() { + origin.hashCode() + prevHash.hashCode() + deposit.hashCode() - + timestamp.hashCode() - ).hashCode(); + + timestamp.hashCode(); } @Override diff --git a/actuator/src/main/java/org/tron/core/vm/repository/Repository.java b/actuator/src/main/java/org/tron/core/vm/repository/Repository.java index 664ee26ee92..8f91d59d0b8 100644 --- a/actuator/src/main/java/org/tron/core/vm/repository/Repository.java +++ b/actuator/src/main/java/org/tron/core/vm/repository/Repository.java @@ -55,6 +55,10 @@ public interface Repository { void updateContractState(byte[] address, ContractStateCapsule contractStateCapsule); + void putNewContract(byte[] address); + + boolean isNewContract(byte[] address); + void updateAccount(byte[] address, AccountCapsule accountCapsule); void updateDynamicProperty(byte[] word, BytesCapsule bytesCapsule); diff --git a/actuator/src/main/java/org/tron/core/vm/repository/RepositoryImpl.java b/actuator/src/main/java/org/tron/core/vm/repository/RepositoryImpl.java index a064cbf6c8a..9de7c0691ba 100644 --- a/actuator/src/main/java/org/tron/core/vm/repository/RepositoryImpl.java +++ b/actuator/src/main/java/org/tron/core/vm/repository/RepositoryImpl.java @@ -9,6 +9,7 @@ import com.google.common.collect.HashBasedTable; import com.google.protobuf.ByteString; import java.util.HashMap; +import java.util.HashSet; import java.util.Optional; import lombok.Getter; import lombok.extern.slf4j.Slf4j; @@ -135,6 +136,7 @@ public class RepositoryImpl implements Repository { private final HashMap> delegationCache = new HashMap<>(); private final HashMap> delegatedResourceAccountIndexCache = new HashMap<>(); private final HashBasedTable> transientStorage = HashBasedTable.create(); + private final HashSet newContractCache = new HashSet<>(); public static void removeLruCache(byte[] address) { } @@ -479,6 +481,7 @@ public void deleteContract(byte[] address) { public void createContract(byte[] address, ContractCapsule contractCapsule) { contractCache.put(Key.create(address), Value.create(contractCapsule, Type.CREATE)); + putNewContract(address); } @Override @@ -533,6 +536,29 @@ public void updateContractState(byte[] address, ContractStateCapsule contractSta Value.create(contractStateCapsule, Type.DIRTY)); } + @Override + public void putNewContract(byte[] address) { + newContractCache.add(Key.create(address)); + } + + @Override + public boolean isNewContract(byte[] address) { + Key key = Key.create(address); + if (newContractCache.contains(key)) { + return true; + } + + if (parent != null) { + boolean isNew = parent.isNewContract(address); + if (isNew) { + newContractCache.add(key); + } + return isNew; + } else { + return false; + } + } + @Override public void updateAccount(byte[] address, AccountCapsule accountCapsule) { accountCache.put(Key.create(address), @@ -740,6 +766,7 @@ public void commit() { commitDelegationCache(repository); commitDelegatedResourceAccountIndexCache(repository); commitTransientStorage(repository); + commitNewContractCache(repository); } @Override @@ -1060,6 +1087,12 @@ public void commitTransientStorage(Repository deposit) { } } + public void commitNewContractCache(Repository deposit) { + if (deposit != null) { + newContractCache.forEach(key -> deposit.putNewContract(key.getData())); + } + } + /** * Get the block id from the number. */ diff --git a/actuator/src/main/java/org/tron/core/vm/repository/Type.java b/actuator/src/main/java/org/tron/core/vm/repository/Type.java index e0842e6a593..9dfbd69f9ae 100644 --- a/actuator/src/main/java/org/tron/core/vm/repository/Type.java +++ b/actuator/src/main/java/org/tron/core/vm/repository/Type.java @@ -73,7 +73,7 @@ public boolean equals(Object obj) { @Override public int hashCode() { - return new Integer(type).hashCode(); + return type; } @Override diff --git a/actuator/src/main/java/org/tron/core/vm/repository/Value.java b/actuator/src/main/java/org/tron/core/vm/repository/Value.java index bf5d99c9c94..1df758f0b3e 100644 --- a/actuator/src/main/java/org/tron/core/vm/repository/Value.java +++ b/actuator/src/main/java/org/tron/core/vm/repository/Value.java @@ -58,6 +58,6 @@ public boolean equals(Object obj) { @Override public int hashCode() { - return new Integer(type.hashCode() + Objects.hashCode(value)).hashCode(); + return type.hashCode() + Objects.hashCode(value); } } diff --git a/actuator/src/main/java/org/tron/core/vm/repository/WriteOptionsWrapper.java b/actuator/src/main/java/org/tron/core/vm/repository/WriteOptionsWrapper.java deleted file mode 100644 index f9e819f9716..00000000000 --- a/actuator/src/main/java/org/tron/core/vm/repository/WriteOptionsWrapper.java +++ /dev/null @@ -1,24 +0,0 @@ -package org.tron.core.vm.repository; - -import lombok.Getter; - -public class WriteOptionsWrapper { - - @Getter - private org.rocksdb.WriteOptions rocks = null; - @Getter - private org.iq80.leveldb.WriteOptions level = null; - - public static WriteOptionsWrapper getInstance() { - WriteOptionsWrapper wrapper = new WriteOptionsWrapper(); - wrapper.level = new org.iq80.leveldb.WriteOptions(); - wrapper.rocks = new org.rocksdb.WriteOptions(); - return wrapper; - } - - public WriteOptionsWrapper sync(boolean bool) { - this.level.sync(bool); - this.rocks.setSync(bool); - return this; - } -} diff --git a/build.gradle b/build.gradle index 14b095b1795..12a0622db99 100644 --- a/build.gradle +++ b/build.gradle @@ -1,14 +1,58 @@ +import org.gradle.nativeplatform.platform.internal.Architectures +import org.gradle.internal.os.OperatingSystem allprojects { version = "1.0.0" apply plugin: "java-library" + ext { + springVersion = "5.3.39" + } +} +def arch = System.getProperty("os.arch").toLowerCase() +def javaVersion = JavaVersion.current() +def isArm64 = Architectures.AARCH64.isAlias(arch) +def archSource = isArm64 ? "arm" : "x86" +def isMac = OperatingSystem.current().isMacOsX() + +ext.archInfo = [ + name : arch, + java : javaVersion, + isArm64 : isArm64, + sourceSets: [ + main: [ + java: [ + srcDirs: ["src/main/java/common", "src/main/java/${archSource}"] + ] + ], + test: [ + java: [ + srcDirs: ["src/test/java"] + ] + ] + ], + requires: [ + JavaVersion: isArm64 ? JavaVersion.VERSION_17 : JavaVersion.VERSION_1_8, + RocksdbVersion: isArm64 ? '9.7.4' : '5.15.10', + // https://github.com/grpc/grpc-java/issues/7690 + // https://github.com/grpc/grpc-java/pull/12319, Add support for macOS aarch64 with universal binary + // https://github.com/grpc/grpc-java/pull/11371 , 1.64.x is not supported CentOS 7. + ProtocGenVersion: isArm64 && isMac ? '1.76.0' : '1.60.0' + ], + VMOptions: isArm64 ? "${rootDir}/gradle/jdk17/java-tron.vmoptions" : "${rootDir}/gradle/java-tron.vmoptions" +] + +if (!archInfo.java.is(archInfo.requires.JavaVersion)) { + throw new GradleException("Java ${archInfo.requires.JavaVersion} is required for ${archInfo.name}. Detected version ${archInfo.java}") } +println "Building for architecture: ${archInfo.name}, Java version: ${archInfo.java}" + + subprojects { apply plugin: "jacoco" apply plugin: "maven-publish" sourceCompatibility = JavaVersion.VERSION_1_8 - targetCompatibility = JavaVersion.VERSION_1_8 + targetCompatibility = JavaVersion.current() [compileJava, compileTestJava]*.options*.encoding = 'UTF-8' jacoco { @@ -41,18 +85,23 @@ subprojects { implementation group: 'org.slf4j', name: 'jcl-over-slf4j', version: '1.7.25' implementation group: 'ch.qos.logback', name: 'logback-classic', version: '1.2.13' implementation "com.google.code.findbugs:jsr305:3.0.0" - implementation group: 'org.springframework', name: 'spring-context', version: '5.3.18' - implementation group: 'org.springframework', name: 'spring-tx', version: '5.3.18' + implementation group: 'org.springframework', name: 'spring-context', version: "${springVersion}" implementation "org.apache.commons:commons-lang3:3.4" implementation group: 'org.apache.commons', name: 'commons-math', version: '2.2' implementation "org.apache.commons:commons-collections4:4.1" implementation group: 'joda-time', name: 'joda-time', version: '2.3' - implementation group: 'org.bouncycastle', name: 'bcprov-jdk15on', version: '1.69' + implementation group: 'org.bouncycastle', name: 'bcprov-jdk18on', version: '1.79' - compileOnly 'org.projectlombok:lombok:1.18.12' - annotationProcessor 'org.projectlombok:lombok:1.18.12' - testCompileOnly 'org.projectlombok:lombok:1.18.12' - testAnnotationProcessor 'org.projectlombok:lombok:1.18.12' + compileOnly 'org.projectlombok:lombok:1.18.34' + annotationProcessor 'org.projectlombok:lombok:1.18.34' + testCompileOnly 'org.projectlombok:lombok:1.18.34' + testAnnotationProcessor 'org.projectlombok:lombok:1.18.34' + + // https://www.oracle.com/java/technologies/javase/11-relnote-issues.html#JDK-8190378 + implementation group: 'javax.annotation', name: 'javax.annotation-api', version: '1.3.2' + // for json-rpc, see https://github.com/briandilley/jsonrpc4j/issues/278 + implementation group: 'javax.jws', name: 'javax.jws-api', version: '1.1' + annotationProcessor group: 'javax.annotation', name: 'javax.annotation-api', version: '1.3.2' testImplementation group: 'junit', name: 'junit', version: '4.13.2' testImplementation "org.mockito:mockito-core:4.11.0" @@ -71,6 +120,33 @@ subprojects { reproducibleFileOrder = true duplicatesStrategy = DuplicatesStrategy.INCLUDE // allow duplicates } + tasks.withType(Test).configureEach { + // https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.Test.html#org.gradle.api.tasks.testing.Test:environment + environment 'CI', 'true' + } + + publishing { + publications { + mavenJava(MavenPublication) { + from components.java + } + } + } + configurations.configureEach { + resolutionStrategy { + eachDependency { details -> + if (details.requested.group == 'com.google.guava' && + details.requested.name == 'guava') { + def requestedVersion = details.requested.version + if (requestedVersion.matches(/.*-android$/)) { + def jreVersion = requestedVersion.replaceAll(/-android$/, '-jre') + details.useVersion(jreVersion) + details.because("Automatically replace android guava with jre version: ${requestedVersion} -> ${jreVersion}") + } + } + } + } + } } task copyToParent(type: Copy) { diff --git a/chainbase/build.gradle b/chainbase/build.gradle index bc82d9496c3..1a07ff95fa5 100644 --- a/chainbase/build.gradle +++ b/chainbase/build.gradle @@ -10,7 +10,6 @@ dependencies { api project(":common") api project(":crypto") api "org.fusesource.jansi:jansi:$jansiVersion" - api 'io.github.tronprotocol:zksnark-java-sdk:1.0.0' api 'org.reflections:reflections:0.9.11' } diff --git a/chainbase/src/main/java/org/tron/common/storage/WriteOptionsWrapper.java b/chainbase/src/main/java/org/tron/common/storage/WriteOptionsWrapper.java index 11277eafe75..bd6cacc6481 100644 --- a/chainbase/src/main/java/org/tron/common/storage/WriteOptionsWrapper.java +++ b/chainbase/src/main/java/org/tron/common/storage/WriteOptionsWrapper.java @@ -1,6 +1,8 @@ package org.tron.common.storage; -public class WriteOptionsWrapper { +import java.io.Closeable; + +public class WriteOptionsWrapper implements Closeable { public org.rocksdb.WriteOptions rocks = null; public org.iq80.leveldb.WriteOptions level = null; @@ -9,6 +11,23 @@ private WriteOptionsWrapper() { } + /** + * Returns an WriteOptionsWrapper. + * + *

CRITICAL: The returned WriteOptionsWrapper holds native resources + * and MUST be closed + * after use to prevent memory leaks. It is strongly recommended to use a try-with-resources + * statement. + * + *

Example of correct usage: + *

{@code
+   * try ( WriteOptionsWrapper readOptions = WriteOptionsWrapper.getInstance()) {
+   *  // do something
+   * }
+   * }
+ * + * @return a new WriteOptionsWrapper that must be closed. + */ public static WriteOptionsWrapper getInstance() { WriteOptionsWrapper wrapper = new WriteOptionsWrapper(); wrapper.level = new org.iq80.leveldb.WriteOptions(); @@ -23,4 +42,12 @@ public WriteOptionsWrapper sync(boolean bool) { this.rocks.setSync(bool); return this; } + + @Override + public void close() { + if (rocks != null) { + rocks.close(); + } + // leveldb WriteOptions has no close method, and does not need to be closed + } } diff --git a/chainbase/src/main/java/org/tron/common/storage/leveldb/LevelDbDataSourceImpl.java b/chainbase/src/main/java/org/tron/common/storage/leveldb/LevelDbDataSourceImpl.java index 506ecdcb6c7..c48800573e1 100644 --- a/chainbase/src/main/java/org/tron/common/storage/leveldb/LevelDbDataSourceImpl.java +++ b/chainbase/src/main/java/org/tron/common/storage/leveldb/LevelDbDataSourceImpl.java @@ -17,8 +17,9 @@ import static org.fusesource.leveldbjni.JniDBFactory.factory; +import com.google.common.annotations.VisibleForTesting; import com.google.common.collect.Sets; -import java.io.File; +import com.google.common.primitives.Bytes; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; @@ -30,26 +31,20 @@ import java.util.List; import java.util.Map; import java.util.Map.Entry; -import java.util.Objects; import java.util.Set; import java.util.concurrent.locks.ReadWriteLock; import java.util.concurrent.locks.ReentrantReadWriteLock; import java.util.stream.Collectors; import java.util.stream.Stream; import java.util.stream.StreamSupport; - -import com.google.common.primitives.Bytes; import lombok.NoArgsConstructor; import lombok.extern.slf4j.Slf4j; -import org.iq80.leveldb.CompressionType; import org.iq80.leveldb.DB; import org.iq80.leveldb.DBIterator; -import org.iq80.leveldb.Logger; import org.iq80.leveldb.Options; import org.iq80.leveldb.ReadOptions; import org.iq80.leveldb.WriteBatch; import org.iq80.leveldb.WriteOptions; -import org.slf4j.LoggerFactory; import org.tron.common.parameter.CommonParameter; import org.tron.common.storage.WriteOptionsWrapper; import org.tron.common.storage.metric.DbStat; @@ -73,29 +68,11 @@ public class LevelDbDataSourceImpl extends DbStat implements DbSourceInter allKeys() { resetDbLock.readLock().lock(); @@ -243,6 +224,7 @@ public Set allKeys() { } @Deprecated + @VisibleForTesting @Override public Set allValues() { resetDbLock.readLock().lock(); @@ -362,6 +344,8 @@ public Map prefixQuery(byte[] key) { } } + @Deprecated + @VisibleForTesting @Override public long getTotal() throws RuntimeException { resetDbLock.readLock().lock(); @@ -378,13 +362,6 @@ public long getTotal() throws RuntimeException { } } - private void updateByBatchInner(Map rows) throws Exception { - try (WriteBatch batch = database.createWriteBatch()) { - innerBatchUpdate(rows,batch); - database.write(batch, writeOptions); - } - } - private void updateByBatchInner(Map rows, WriteOptions options) throws Exception { try (WriteBatch batch = database.createWriteBatch()) { innerBatchUpdate(rows,batch); @@ -404,30 +381,23 @@ private void innerBatchUpdate(Map rows, WriteBatch batch) { @Override public void updateByBatch(Map rows, WriteOptionsWrapper options) { - resetDbLock.readLock().lock(); - try { - updateByBatchInner(rows, options.level); - } catch (Exception e) { - try { - updateByBatchInner(rows, options.level); - } catch (Exception e1) { - throw new RuntimeException(e); - } - } finally { - resetDbLock.readLock().unlock(); - } + this.updateByBatch(rows, options.level); } @Override public void updateByBatch(Map rows) { + this.updateByBatch(rows, writeOptions); + } + + private void updateByBatch(Map rows, WriteOptions options) { resetDbLock.readLock().lock(); try { - updateByBatchInner(rows); + updateByBatchInner(rows, options); } catch (Exception e) { try { - updateByBatchInner(rows); + updateByBatchInner(rows, options); } catch (Exception e1) { - throw new RuntimeException(e); + throw new RuntimeException(e1); } } finally { resetDbLock.readLock().unlock(); @@ -455,6 +425,24 @@ public void closeDB() { } } + /** + * Returns an iterator over the database. + * + *

CRITICAL: The returned iterator holds native resources and MUST be closed + * after use to prevent memory leaks. It is strongly recommended to use a try-with-resources + * statement. + * + *

Example of correct usage: + *

{@code
+   * try (DBIterator iterator = db.iterator()) {
+   *   while (iterator.hasNext()) {
+   *     // ... process entry
+   *   }
+   * }
+   * }
+ * + * @return a new database iterator that must be closed. + */ @Override public org.tron.core.db.common.iterator.DBIterator iterator() { return new StoreIterator(getDBIterator()); @@ -467,7 +455,7 @@ public Stream> stream() { @Override public LevelDbDataSourceImpl newInstance() { return new LevelDbDataSourceImpl(StorageUtils.getOutputDirectoryByDbName(dataBaseName), - dataBaseName, options, writeOptions); + dataBaseName); } private DBIterator getDBIterator() { diff --git a/chainbase/src/main/java/org/tron/common/storage/rocksdb/RocksDbDataSourceImpl.java b/chainbase/src/main/java/org/tron/common/storage/rocksdb/RocksDbDataSourceImpl.java index 5c051bdc101..c7ca698cc3d 100644 --- a/chainbase/src/main/java/org/tron/common/storage/rocksdb/RocksDbDataSourceImpl.java +++ b/chainbase/src/main/java/org/tron/common/storage/rocksdb/RocksDbDataSourceImpl.java @@ -1,5 +1,6 @@ package org.tron.common.storage.rocksdb; +import com.google.common.annotations.VisibleForTesting; import com.google.common.collect.Sets; import com.google.common.primitives.Bytes; import java.io.File; @@ -20,27 +21,20 @@ import java.util.stream.Collectors; import lombok.NoArgsConstructor; import lombok.extern.slf4j.Slf4j; -import org.rocksdb.BlockBasedTableConfig; -import org.rocksdb.BloomFilter; import org.rocksdb.Checkpoint; -import org.rocksdb.DirectComparator; -import org.rocksdb.InfoLogLevel; -import org.rocksdb.Logger; import org.rocksdb.Options; import org.rocksdb.ReadOptions; import org.rocksdb.RocksDB; import org.rocksdb.RocksDBException; import org.rocksdb.RocksIterator; -import org.rocksdb.Statistics; import org.rocksdb.Status; import org.rocksdb.WriteBatch; import org.rocksdb.WriteOptions; -import org.slf4j.LoggerFactory; +import org.tron.common.error.TronDBException; import org.tron.common.setting.RocksDbSettings; import org.tron.common.storage.WriteOptionsWrapper; import org.tron.common.storage.metric.DbStat; import org.tron.common.utils.FileUtil; -import org.tron.common.utils.PropUtil; import org.tron.core.db.common.DbSourceInter; import org.tron.core.db.common.iterator.RockStoreIterator; import org.tron.core.db2.common.Instance; @@ -53,38 +47,19 @@ public class RocksDbDataSourceImpl extends DbStat implements DbSourceInter, Iterable>, Instance { - ReadOptions readOpts; private String dataBaseName; private RocksDB database; private volatile boolean alive; private String parentPath; private ReadWriteLock resetDbLock = new ReentrantReadWriteLock(); - private static final String KEY_ENGINE = "ENGINE"; - private static final String ROCKSDB = "ROCKSDB"; - private DirectComparator comparator; - private static final org.slf4j.Logger rocksDbLogger = LoggerFactory.getLogger(ROCKSDB); + private Options options; - public RocksDbDataSourceImpl(String parentPath, String name, RocksDbSettings settings, - DirectComparator comparator) { - this.dataBaseName = name; - this.parentPath = parentPath; - this.comparator = comparator; - RocksDbSettings.setRocksDbSettings(settings); - initDB(); - } - - public RocksDbDataSourceImpl(String parentPath, String name, RocksDbSettings settings) { + public RocksDbDataSourceImpl(String parentPath, String name) { this.dataBaseName = name; this.parentPath = parentPath; - RocksDbSettings.setRocksDbSettings(settings); initDB(); } - public RocksDbDataSourceImpl(String parentPath, String name) { - this.parentPath = parentPath; - this.dataBaseName = name; - } - public Path getDbPath() { return Paths.get(parentPath, dataBaseName); } @@ -104,6 +79,9 @@ public void closeDB() { if (!isAlive()) { return; } + if (this.options != null) { + this.options.close(); + } database.close(); alive = false; } catch (Exception e) { @@ -125,40 +103,68 @@ public void resetDb() { } } - private boolean quitIfNotAlive() { + private void throwIfNotAlive() { if (!isAlive()) { - logger.warn("DB {} is not alive.", dataBaseName); + throw new TronDBException("DB " + this.getDBName() + " is closed."); } - return !isAlive(); } + /** copy from {@link org.fusesource.leveldbjni.internal#checkArgNotNull} */ + private static void checkArgNotNull(Object value, String name) { + if (value == null) { + throw new IllegalArgumentException("The " + name + " argument cannot be null"); + } + } + + @Deprecated + @VisibleForTesting @Override public Set allKeys() throws RuntimeException { resetDbLock.readLock().lock(); - try { - if (quitIfNotAlive()) { - return null; - } + try (final ReadOptions readOptions = getReadOptions(); + final RocksIterator iter = getRocksIterator(readOptions)) { Set result = Sets.newHashSet(); - try (final RocksIterator iter = getRocksIterator()) { - for (iter.seekToFirst(); iter.isValid(); iter.next()) { - result.add(iter.key()); - } - return result; + for (iter.seekToFirst(); iter.isValid(); iter.next()) { + result.add(iter.key()); } + return result; } finally { resetDbLock.readLock().unlock(); } } + @Deprecated + @VisibleForTesting @Override public Set allValues() throws RuntimeException { - return null; + resetDbLock.readLock().lock(); + try (final ReadOptions readOptions = getReadOptions(); + final RocksIterator iter = getRocksIterator(readOptions)) { + Set result = Sets.newHashSet(); + for (iter.seekToFirst(); iter.isValid(); iter.next()) { + result.add(iter.value()); + } + return result; + } finally { + resetDbLock.readLock().unlock(); + } } + @Deprecated + @VisibleForTesting @Override public long getTotal() throws RuntimeException { - return 0; + resetDbLock.readLock().lock(); + try (final ReadOptions readOptions = getReadOptions(); + final RocksIterator iter = getRocksIterator(readOptions)) { + long total = 0; + for (iter.seekToFirst(); iter.isValid(); iter.next()) { + total++; + } + return total; + } finally { + resetDbLock.readLock().unlock(); + } } @Override @@ -168,39 +174,10 @@ public String getDBName() { @Override public void setDBName(String name) { + this.dataBaseName = name; } - public boolean checkOrInitEngine() { - String dir = getDbPath().toString(); - String enginePath = dir + File.separator + "engine.properties"; - - if (FileUtil.createDirIfNotExists(dir)) { - if (!FileUtil.createFileIfNotExists(enginePath)) { - return false; - } - } else { - return false; - } - - // for the first init engine - String engine = PropUtil.readProperty(enginePath, KEY_ENGINE); - if (engine.isEmpty() && !PropUtil.writeProperty(enginePath, KEY_ENGINE, ROCKSDB)) { - return false; - } - engine = PropUtil.readProperty(enginePath, KEY_ENGINE); - - return ROCKSDB.equals(engine); - } - - public void initDB() { - if (!checkOrInitEngine()) { - throw new RuntimeException( - String.format("failed to check database: %s, engine do not match", dataBaseName)); - } - initDB(RocksDbSettings.getSettings()); - } - - public void initDB(RocksDbSettings settings) { + private void initDB() { resetDbLock.writeLock().lock(); try { if (isAlive()) { @@ -210,80 +187,40 @@ public void initDB(RocksDbSettings settings) { throw new IllegalArgumentException("No name set to the dbStore"); } - try (Options options = new Options()) { - - // most of these options are suggested by https://github.com/facebook/rocksdb/wiki/Set-Up-Options + try { + logger.debug("Opening database {}.", dataBaseName); + final Path dbPath = getDbPath(); - // general options - if (settings.isEnableStatistics()) { - options.setStatistics(new Statistics()); - options.setStatsDumpPeriodSec(60); + if (!Files.isSymbolicLink(dbPath.getParent())) { + Files.createDirectories(dbPath.getParent()); } - options.setCreateIfMissing(true); - options.setIncreaseParallelism(1); - options.setLevelCompactionDynamicLevelBytes(true); - options.setMaxOpenFiles(settings.getMaxOpenFiles()); - - // general options supported user config - options.setNumLevels(settings.getLevelNumber()); - options.setMaxBytesForLevelMultiplier(settings.getMaxBytesForLevelMultiplier()); - options.setMaxBytesForLevelBase(settings.getMaxBytesForLevelBase()); - options.setMaxBackgroundCompactions(settings.getCompactThreads()); - options.setLevel0FileNumCompactionTrigger(settings.getLevel0FileNumCompactionTrigger()); - options.setTargetFileSizeMultiplier(settings.getTargetFileSizeMultiplier()); - options.setTargetFileSizeBase(settings.getTargetFileSizeBase()); - if (comparator != null) { - options.setComparator(comparator); - } - options.setLogger(new Logger(options) { - @Override - protected void log(InfoLogLevel infoLogLevel, String logMsg) { - rocksDbLogger.info("{} {}", dataBaseName, logMsg); - } - }); - - // table options - final BlockBasedTableConfig tableCfg; - options.setTableFormatConfig(tableCfg = new BlockBasedTableConfig()); - tableCfg.setBlockSize(settings.getBlockSize()); - tableCfg.setBlockCache(RocksDbSettings.getCache()); - tableCfg.setCacheIndexAndFilterBlocks(true); - tableCfg.setPinL0FilterAndIndexBlocksInCache(true); - tableCfg.setFilter(new BloomFilter(10, false)); - - // read options - readOpts = new ReadOptions(); - readOpts = readOpts.setPrefixSameAsStart(true) - .setVerifyChecksums(false); try { - logger.debug("Opening database {}.", dataBaseName); - final Path dbPath = getDbPath(); - - if (!Files.isSymbolicLink(dbPath.getParent())) { - Files.createDirectories(dbPath.getParent()); + DbSourceInter.checkOrInitEngine(getEngine(), dbPath.toString(), + TronError.ErrCode.ROCKSDB_INIT); + this.options = RocksDbSettings.getOptionsByDbName(dataBaseName); + database = RocksDB.open(this.options, dbPath.toString()); + } catch (RocksDBException e) { + if (Objects.equals(e.getStatus().getCode(), Status.Code.Corruption)) { + logger.error("Database {} corrupted, please delete database directory({}) " + + "and restart.", dataBaseName, parentPath, e); + } else { + logger.error("Open Database {} failed", dataBaseName, e); } - try { - database = RocksDB.open(options, dbPath.toString()); - } catch (RocksDBException e) { - if (Objects.equals(e.getStatus().getCode(), Status.Code.Corruption)) { - logger.error("Database {} corrupted, please delete database directory({}) " + - "and restart.", dataBaseName, parentPath, e); - } else { - logger.error("Open Database {} failed", dataBaseName, e); - } - throw new TronError(e, TronError.ErrCode.ROCKSDB_INIT); + if (this.options != null) { + this.options.close(); } - - alive = true; - } catch (IOException ioe) { - throw new RuntimeException( - String.format("failed to init database: %s", dataBaseName), ioe); + throw new TronError(e, TronError.ErrCode.ROCKSDB_INIT); } - logger.debug("Init DB {} done.", dataBaseName); + alive = true; + } catch (IOException ioe) { + throw new RuntimeException( + String.format("failed to init database: %s", dataBaseName), ioe); } + + logger.debug("Init DB {} done.", dataBaseName); } finally { resetDbLock.writeLock().unlock(); } @@ -293,9 +230,9 @@ protected void log(InfoLogLevel infoLogLevel, String logMsg) { public void putData(byte[] key, byte[] value) { resetDbLock.readLock().lock(); try { - if (quitIfNotAlive()) { - return; - } + throwIfNotAlive(); + checkArgNotNull(key, "key"); + checkArgNotNull(value, "value"); database.put(key, value); } catch (RocksDBException e) { throw new RuntimeException(dataBaseName, e); @@ -308,9 +245,8 @@ public void putData(byte[] key, byte[] value) { public byte[] getData(byte[] key) { resetDbLock.readLock().lock(); try { - if (quitIfNotAlive()) { - return null; - } + throwIfNotAlive(); + checkArgNotNull(key, "key"); return database.get(key); } catch (RocksDBException e) { throw new RuntimeException(dataBaseName, e); @@ -323,9 +259,8 @@ public byte[] getData(byte[] key) { public void deleteData(byte[] key) { resetDbLock.readLock().lock(); try { - if (quitIfNotAlive()) { - return; - } + throwIfNotAlive(); + checkArgNotNull(key, "key"); database.delete(key); } catch (RocksDBException e) { throw new RuntimeException(dataBaseName, e); @@ -339,74 +274,65 @@ public boolean flush() { return false; } + /** + * Returns an iterator over the database. + * + *

CRITICAL: The returned iterator holds native resources and MUST be closed + * after use to prevent memory leaks. It is strongly recommended to use a try-with-resources + * statement. + * + *

Example of correct usage: + *

{@code
+   * try (DBIterator iterator = db.iterator()) {
+   *   while (iterator.hasNext()) {
+   *     // ... process entry
+   *   }
+   * }
+   * }
+ * + * @return a new database iterator that must be closed. + */ @Override public org.tron.core.db.common.iterator.DBIterator iterator() { - return new RockStoreIterator(getRocksIterator()); - } - - private void updateByBatchInner(Map rows) throws Exception { - if (quitIfNotAlive()) { - return; - } - try (WriteBatch batch = new WriteBatch()) { - for (Map.Entry entry : rows.entrySet()) { - if (entry.getValue() == null) { - batch.delete(entry.getKey()); - } else { - batch.put(entry.getKey(), entry.getValue()); - } - } - database.write(new WriteOptions(), batch); - } + ReadOptions readOptions = getReadOptions(); + return new RockStoreIterator(getRocksIterator(readOptions), readOptions); } private void updateByBatchInner(Map rows, WriteOptions options) throws Exception { - if (quitIfNotAlive()) { - return; - } try (WriteBatch batch = new WriteBatch()) { for (Map.Entry entry : rows.entrySet()) { + checkArgNotNull(entry.getKey(), "key"); if (entry.getValue() == null) { batch.delete(entry.getKey()); } else { batch.put(entry.getKey(), entry.getValue()); } } + throwIfNotAlive(); database.write(options, batch); } } @Override public void updateByBatch(Map rows, WriteOptionsWrapper optionsWrapper) { - resetDbLock.readLock().lock(); - try { - if (quitIfNotAlive()) { - return; - } - updateByBatchInner(rows, optionsWrapper.rocks); - } catch (Exception e) { - try { - updateByBatchInner(rows); - } catch (Exception e1) { - throw new RuntimeException(dataBaseName, e1); - } - } finally { - resetDbLock.readLock().unlock(); - } + this.updateByBatch(rows, optionsWrapper.rocks); } @Override public void updateByBatch(Map rows) { + try (WriteOptions writeOptions = new WriteOptions()) { + this.updateByBatch(rows, writeOptions); + } + } + + private void updateByBatch(Map rows, WriteOptions options) { resetDbLock.readLock().lock(); try { - if (quitIfNotAlive()) { - return; - } - updateByBatchInner(rows); + updateByBatchInner(rows, options); } catch (Exception e) { try { - updateByBatchInner(rows); + updateByBatchInner(rows, options); } catch (Exception e1) { throw new RuntimeException(dataBaseName, e1); } @@ -416,45 +342,36 @@ public void updateByBatch(Map rows) { } public List getKeysNext(byte[] key, long limit) { + if (limit <= 0) { + return new ArrayList<>(); + } resetDbLock.readLock().lock(); - try { - if (quitIfNotAlive()) { - return new ArrayList<>(); - } - if (limit <= 0) { - return new ArrayList<>(); - } - - try (RocksIterator iter = getRocksIterator()) { - List result = new ArrayList<>(); - long i = 0; - for (iter.seek(key); iter.isValid() && i < limit; iter.next(), i++) { - result.add(iter.key()); - } - return result; + try (final ReadOptions readOptions = getReadOptions(); + final RocksIterator iter = getRocksIterator(readOptions)) { + List result = new ArrayList<>(); + long i = 0; + for (iter.seek(key); iter.isValid() && i < limit; iter.next(), i++) { + result.add(iter.key()); } + return result; } finally { resetDbLock.readLock().unlock(); } } public Map getNext(byte[] key, long limit) { + if (limit <= 0) { + return Collections.emptyMap(); + } resetDbLock.readLock().lock(); - try { - if (quitIfNotAlive()) { - return null; - } - if (limit <= 0) { - return Collections.emptyMap(); - } - try (RocksIterator iter = getRocksIterator()) { - Map result = new HashMap<>(); - long i = 0; - for (iter.seek(key); iter.isValid() && i < limit; iter.next(), i++) { - result.put(iter.key(), iter.value()); - } - return result; + try (final ReadOptions readOptions = getReadOptions(); + final RocksIterator iter = getRocksIterator(readOptions)) { + Map result = new HashMap<>(); + long i = 0; + for (iter.seek(key); iter.isValid() && i < limit; iter.next(), i++) { + result.put(iter.key(), iter.value()); } + return result; } finally { resetDbLock.readLock().unlock(); } @@ -463,80 +380,109 @@ public Map getNext(byte[] key, long limit) { @Override public Map prefixQuery(byte[] key) { resetDbLock.readLock().lock(); - try { - if (quitIfNotAlive()) { - return null; - } - try (RocksIterator iterator = getRocksIterator()) { - Map result = new HashMap<>(); - for (iterator.seek(key); iterator.isValid(); iterator.next()) { - if (Bytes.indexOf(iterator.key(), key) == 0) { - result.put(WrappedByteArray.of(iterator.key()), iterator.value()); - } else { - return result; - } + try (final ReadOptions readOptions = getReadOptions(); + final RocksIterator iterator = getRocksIterator(readOptions)) { + Map result = new HashMap<>(); + for (iterator.seek(key); iterator.isValid(); iterator.next()) { + if (Bytes.indexOf(iterator.key(), key) == 0) { + result.put(WrappedByteArray.of(iterator.key()), iterator.value()); + } else { + return result; } - return result; } + return result; } finally { resetDbLock.readLock().unlock(); } } public Set getlatestValues(long limit) { + if (limit <= 0) { + return Sets.newHashSet(); + } resetDbLock.readLock().lock(); - try { - if (quitIfNotAlive()) { - return null; - } - if (limit <= 0) { - return Sets.newHashSet(); - } - try (RocksIterator iter = getRocksIterator()) { - Set result = Sets.newHashSet(); - long i = 0; - for (iter.seekToLast(); iter.isValid() && i < limit; iter.prev(), i++) { - result.add(iter.value()); - } - return result; + try (final ReadOptions readOptions = getReadOptions(); + final RocksIterator iter = getRocksIterator(readOptions)) { + Set result = Sets.newHashSet(); + long i = 0; + for (iter.seekToLast(); iter.isValid() && i < limit; iter.prev(), i++) { + result.add(iter.value()); } + return result; } finally { resetDbLock.readLock().unlock(); } } - public Set getValuesNext(byte[] key, long limit) { + if (limit <= 0) { + return Sets.newHashSet(); + } resetDbLock.readLock().lock(); - try { - if (quitIfNotAlive()) { - return null; - } - if (limit <= 0) { - return Sets.newHashSet(); - } - try (RocksIterator iter = getRocksIterator()) { - Set result = Sets.newHashSet(); - long i = 0; - for (iter.seek(key); iter.isValid() && i < limit; iter.next(), i++) { - result.add(iter.value()); - } - return result; + try (final ReadOptions readOptions = getReadOptions(); + final RocksIterator iter = getRocksIterator(readOptions)) { + Set result = Sets.newHashSet(); + long i = 0; + for (iter.seek(key); iter.isValid() && i < limit; iter.next(), i++) { + result.add(iter.value()); } + return result; } finally { resetDbLock.readLock().unlock(); } } public void backup(String dir) throws RocksDBException { - Checkpoint cp = Checkpoint.create(database); - cp.createCheckpoint(dir + this.getDBName()); + throwIfNotAlive(); + try (Checkpoint cp = Checkpoint.create(database)) { + cp.createCheckpoint(dir + this.getDBName()); + } } - private RocksIterator getRocksIterator() { - try ( ReadOptions readOptions = new ReadOptions().setFillCache(false)) { - return database.newIterator(readOptions); - } + /** + * Returns an iterator over the database. + * + *

CRITICAL: The returned iterator holds native resources and MUST be closed + * after use to prevent memory leaks. It is strongly recommended to use a try-with-resources + * statement. + * + *

Example of correct usage: + *

{@code
+   * try ( ReadOptions readOptions = new ReadOptions().setFillCache(false);
+   *      RocksIterator iterator = getRocksIterator(readOptions)) {
+   *      iterator.seekToFirst();
+   *  // do something
+   * }
+   * }
+ * + * @return a new database iterator that must be closed. + */ + private RocksIterator getRocksIterator(ReadOptions readOptions) { + throwIfNotAlive(); + return database.newIterator(readOptions); + } + + /** + * Returns an ReadOptions. + * + *

CRITICAL: The returned ReadOptions holds native resources and MUST be closed + * after use to prevent memory leaks. It is strongly recommended to use a try-with-resources + * statement. + * + *

Example of correct usage: + *

{@code
+   * try (ReadOptions readOptions = getReadOptions();
+   *      RocksIterator iterator = getRocksIterator(readOptions)) {
+   *      iterator.seekToFirst();
+   *  // do something
+   * }
+   * }
+ * + * @return a new database iterator that must be closed. + */ + private ReadOptions getReadOptions() { + throwIfNotAlive(); + return new ReadOptions().setFillCache(false); } public boolean deleteDbBakPath(String dir) { @@ -545,7 +491,7 @@ public boolean deleteDbBakPath(String dir) { @Override public RocksDbDataSourceImpl newInstance() { - return new RocksDbDataSourceImpl(parentPath, dataBaseName, RocksDbSettings.getSettings()); + return new RocksDbDataSourceImpl(parentPath, dataBaseName); } diff --git a/chainbase/src/main/java/org/tron/common/utils/LocalWitnesses.java b/chainbase/src/main/java/org/tron/common/utils/LocalWitnesses.java index 940a107a2ac..7179045ea7e 100644 --- a/chainbase/src/main/java/org/tron/common/utils/LocalWitnesses.java +++ b/chainbase/src/main/java/org/tron/common/utils/LocalWitnesses.java @@ -25,6 +25,7 @@ import org.tron.common.crypto.SignInterface; import org.tron.common.crypto.SignUtils; import org.tron.core.config.Parameter.ChainConstant; +import org.tron.core.exception.TronError; @Slf4j(topic = "app") public class LocalWitnesses { @@ -32,6 +33,7 @@ public class LocalWitnesses { @Getter private List privateKeys = Lists.newArrayList(); + @Getter private byte[] witnessAccountAddress; public LocalWitnesses() { @@ -45,21 +47,11 @@ public LocalWitnesses(List privateKeys) { setPrivateKeys(privateKeys); } - public byte[] getWitnessAccountAddress(boolean isECKeyCryptoEngine) { - if (witnessAccountAddress == null) { - byte[] privateKey = ByteArray.fromHexString(getPrivateKey()); - final SignInterface cryptoEngine = SignUtils.fromPrivate(privateKey, isECKeyCryptoEngine); - this.witnessAccountAddress = cryptoEngine.getAddress(); - } - return witnessAccountAddress; - } - - public void setWitnessAccountAddress(final byte[] localWitnessAccountAddress) { - this.witnessAccountAddress = localWitnessAccountAddress; - } - - public void initWitnessAccountAddress(boolean isECKeyCryptoEngine) { - if (witnessAccountAddress == null) { + public void initWitnessAccountAddress(final byte[] witnessAddress, + boolean isECKeyCryptoEngine) { + if (witnessAddress != null) { + this.witnessAccountAddress = witnessAddress; + } else if (!CollectionUtils.isEmpty(privateKeys)) { byte[] privateKey = ByteArray.fromHexString(getPrivateKey()); final SignInterface ecKey = SignUtils.fromPrivate(privateKey, isECKeyCryptoEngine); @@ -85,11 +77,16 @@ private void validate(String privateKey) { privateKey = privateKey.substring(2); } - if (StringUtils.isNotBlank(privateKey) - && privateKey.length() != ChainConstant.PRIVATE_KEY_LENGTH) { - throw new IllegalArgumentException( - String.format("private key must be %d-bits hex string, actual: %d", - ChainConstant.PRIVATE_KEY_LENGTH, privateKey.length())); + if (StringUtils.isBlank(privateKey) + || privateKey.length() != ChainConstant.PRIVATE_KEY_LENGTH) { + throw new TronError(String.format("private key must be %d hex string, actual: %d", + ChainConstant.PRIVATE_KEY_LENGTH, + StringUtils.isBlank(privateKey) ? 0 : privateKey.length()), + TronError.ErrCode.WITNESS_INIT); + } + if (!StringUtil.isHexadecimal(privateKey)) { + throw new TronError("private key must be hex string", + TronError.ErrCode.WITNESS_INIT); } } diff --git a/chainbase/src/main/java/org/tron/common/utils/StorageUtils.java b/chainbase/src/main/java/org/tron/common/utils/StorageUtils.java index 16df43f1534..0c7c77bd23f 100644 --- a/chainbase/src/main/java/org/tron/common/utils/StorageUtils.java +++ b/chainbase/src/main/java/org/tron/common/utils/StorageUtils.java @@ -1,15 +1,20 @@ package org.tron.common.utils; import static org.tron.common.parameter.CommonParameter.ENERGY_LIMIT_HARD_FORK; +import static org.tron.core.db.common.DbSourceInter.LEVELDB; import java.io.File; import org.apache.commons.lang3.StringUtils; import org.iq80.leveldb.Options; +import org.slf4j.LoggerFactory; import org.tron.common.parameter.CommonParameter; +import org.tron.core.Constant; public class StorageUtils { + private static final org.slf4j.Logger levelDbLogger = LoggerFactory.getLogger(LEVELDB); + public static boolean getEnergyLimitHardFork() { return ENERGY_LIMIT_HARD_FORK; } @@ -52,9 +57,16 @@ public static String getOutputDirectory() { } public static Options getOptionsByDbName(String dbName) { + Options options; if (hasProperty(dbName)) { - return getProperty(dbName).getDbOptions(); + options = getProperty(dbName).getDbOptions(); + } else { + options = CommonParameter.getInstance().getStorage().newDefaultDbOptions(dbName); + } + if (Constant.MARKET_PAIR_PRICE_TO_ORDER.equals(dbName)) { + options.comparator(new MarketOrderPriceComparatorForLevelDB()); } - return CommonParameter.getInstance().getStorage().newDefaultDbOptions(dbName); + options.logger(message -> levelDbLogger.info("{} {}", dbName, message)); + return options; } } diff --git a/chainbase/src/main/java/org/tron/common/zksnark/JLibrustzcash.java b/chainbase/src/main/java/org/tron/common/zksnark/JLibrustzcash.java index f7b8c17b57f..3700d300411 100644 --- a/chainbase/src/main/java/org/tron/common/zksnark/JLibrustzcash.java +++ b/chainbase/src/main/java/org/tron/common/zksnark/JLibrustzcash.java @@ -29,65 +29,42 @@ @Slf4j public class JLibrustzcash { - private static Librustzcash INSTANCE; + private static Librustzcash INSTANCE = LibrustzcashWrapper.getInstance(); public static void librustzcashZip32XskMaster(Zip32XskMasterParams params) { - if (!isOpenZen()) { - return; - } INSTANCE.librustzcashZip32XskMaster(params.getData(), params.getSize(), params.getM_bytes()); } public static void librustzcashInitZksnarkParams(InitZksnarkParams params) { - if (!isOpenZen()) { - return; - } INSTANCE.librustzcashInitZksnarkParams(params.getSpend_path(), params.getSpend_hash(), params.getOutput_path(), params.getOutput_hash()); } public static void librustzcashZip32XskDerive(Zip32XskDeriveParams params) { - if (!isOpenZen()) { - return; - } INSTANCE.librustzcashZip32XskDerive(params.getData(), params.getSize(), params.getM_bytes()); } public static boolean librustzcashZip32XfvkAddress(Zip32XfvkAddressParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashZip32XfvkAddress(params.getXfvk(), params.getJ(), params.getJ_ret(), params.getAddr_ret()); } public static void librustzcashCrhIvk(CrhIvkParams params) { - if (!isOpenZen()) { - return; - } INSTANCE.librustzcashCrhIvk(params.getAk(), params.getNk(), params.getIvk()); } public static boolean librustzcashKaAgree(KaAgreeParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingKaAgree(params.getP(), params.getSk(), params.getResult()); } public static boolean librustzcashComputeCm(ComputeCmParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingComputeCm(params.getD(), params.getPkD(), params.getValue(), params.getR(), params.getCm()); } public static boolean librustzcashComputeNf(ComputeNfParams params) { - if (isOpenZen()) { - INSTANCE.librustzcashSaplingComputeNf(params.getD(), params.getPkD(), params.getValue(), - params.getR(), params.getAk(), params.getNk(), params.getPosition(), params.getResult()); - } + INSTANCE.librustzcashSaplingComputeNf(params.getD(), params.getPkD(), params.getValue(), + params.getR(), params.getAk(), params.getNk(), params.getPosition(), params.getResult()); return true; } @@ -96,9 +73,6 @@ public static boolean librustzcashComputeNf(ComputeNfParams params) { * @return ak 32 bytes */ public static byte[] librustzcashAskToAk(byte[] ask) throws ZksnarkException { - if (!isOpenZen()) { - return ByteUtil.EMPTY_BYTE_ARRAY; - } LibrustzcashParam.valid32Params(ask); byte[] ak = new byte[32]; INSTANCE.librustzcashAskToAk(ask, ak); @@ -110,9 +84,6 @@ public static byte[] librustzcashAskToAk(byte[] ask) throws ZksnarkException { * @return 32 bytes */ public static byte[] librustzcashNskToNk(byte[] nsk) throws ZksnarkException { - if (!isOpenZen()) { - return ByteUtil.EMPTY_BYTE_ARRAY; - } LibrustzcashParam.valid32Params(nsk); byte[] nk = new byte[32]; INSTANCE.librustzcashNskToNk(nsk, nk); @@ -125,26 +96,17 @@ public static byte[] librustzcashNskToNk(byte[] nsk) throws ZksnarkException { * @return r: random number, less than r_J, 32 bytes */ public static byte[] librustzcashSaplingGenerateR(byte[] r) throws ZksnarkException { - if (!isOpenZen()) { - return ByteUtil.EMPTY_BYTE_ARRAY; - } LibrustzcashParam.valid32Params(r); INSTANCE.librustzcashSaplingGenerateR(r); return r; } public static boolean librustzcashSaplingKaDerivepublic(KaDerivepublicParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingKaDerivepublic(params.getDiversifier(), params.getEsk(), params.getResult()); } public static long librustzcashSaplingProvingCtxInit() { - if (!isOpenZen()) { - return 0; - } return INSTANCE.librustzcashSaplingProvingCtxInit(); } @@ -154,17 +116,11 @@ public static long librustzcashSaplingProvingCtxInit() { * @param d 11 bytes */ public static boolean librustzcashCheckDiversifier(byte[] d) throws ZksnarkException { - if (!isOpenZen()) { - return true; - } LibrustzcashParam.valid11Params(d); return INSTANCE.librustzcashCheckDiversifier(d); } public static boolean librustzcashSaplingSpendProof(SpendProofParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingSpendProof(params.getCtx(), params.getAk(), params.getNsk(), params.getD(), params.getR(), params.getAlpha(), params.getValue(), params.getAnchor(), params.getVoucherPath(), params.getCv(), params.getRk(), @@ -172,26 +128,17 @@ public static boolean librustzcashSaplingSpendProof(SpendProofParams params) { } public static boolean librustzcashSaplingOutputProof(OutputProofParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingOutputProof(params.getCtx(), params.getEsk(), params.getD(), params.getPkD(), params.getR(), params.getValue(), params.getCv(), params.getZkproof()); } public static boolean librustzcashSaplingSpendSig(SpendSigParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingSpendSig(params.getAsk(), params.getAlpha(), params.getSigHash(), params.getResult()); } public static boolean librustzcashSaplingBindingSig(BindingSigParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingBindingSig(params.getCtx(), params.getValueBalance(), params.getSighash(), params.getResult()); } @@ -203,74 +150,47 @@ public static boolean librustzcashSaplingBindingSig(BindingSigParams params) { * @param data 32 bytes */ public static void librustzcashToScalar(byte[] value, byte[] data) throws ZksnarkException { - if (!isOpenZen()) { - return; - } LibrustzcashParam.validParamLength(value, 64); LibrustzcashParam.valid32Params(data); INSTANCE.librustzcashToScalar(value, data); } public static void librustzcashSaplingProvingCtxFree(long ctx) { - if (!isOpenZen()) { - return; - } INSTANCE.librustzcashSaplingProvingCtxFree(ctx); } public static long librustzcashSaplingVerificationCtxInit() { - if (!isOpenZen()) { - return 0; - } return INSTANCE.librustzcashSaplingVerificationCtxInit(); } public static boolean librustzcashSaplingCheckSpend(CheckSpendParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingCheckSpend(params.getCtx(), params.getCv(), params.getAnchor(), params.getNullifier(), params.getRk(), params.getZkproof(), params.getSpendAuthSig(), params.getSighashValue()); } public static boolean librustzcashSaplingCheckOutput(CheckOutputParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingCheckOutput(params.getCtx(), params.getCv(), params.getCm(), params.getEphemeralKey(), params.getZkproof()); } public static boolean librustzcashSaplingFinalCheck(FinalCheckParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingFinalCheck(params.getCtx(), params.getValueBalance(), params.getBindingSig(), params.getSighashValue()); } public static boolean librustzcashSaplingCheckSpendNew(CheckSpendNewParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingCheckSpendNew(params.getCv(), params.getAnchor(), params.getNullifier(), params.getRk(), params.getZkproof(), params.getSpendAuthSig(), params.getSighashValue()); } public static boolean librustzcashSaplingCheckOutputNew(CheckOutputNewParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashSaplingCheckOutputNew(params.getCv(), params.getCm(), params.getEphemeralKey(), params.getZkproof()); } public static boolean librustzcashSaplingFinalCheckNew(FinalCheckNewParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE .librustzcashSaplingFinalCheckNew(params.getValueBalance(), params.getBindingSig(), params.getSighashValue(), params.getSpendCv(), params.getSpendCvLen(), @@ -278,23 +198,14 @@ public static boolean librustzcashSaplingFinalCheckNew(FinalCheckNewParams param } public static void librustzcashSaplingVerificationCtxFree(long ctx) { - if (!isOpenZen()) { - return; - } INSTANCE.librustzcashSaplingVerificationCtxFree(ctx); } public static boolean librustzcashIvkToPkd(IvkToPkdParams params) { - if (!isOpenZen()) { - return true; - } return INSTANCE.librustzcashIvkToPkd(params.getIvk(), params.getD(), params.getPkD()); } public static void librustzcashMerkleHash(MerkleHashParams params) { - if (!isOpenZen()) { - return; - } INSTANCE.librustzcashMerkleHash(params.getDepth(), params.getA(), params.getB(), params.getResult()); } @@ -303,19 +214,7 @@ public static void librustzcashMerkleHash(MerkleHashParams params) { * @param result uncommitted value, 32 bytes */ public static void librustzcashTreeUncommitted(byte[] result) throws ZksnarkException { - if (!isOpenZen()) { - return; - } LibrustzcashParam.valid32Params(result); INSTANCE.librustzcashTreeUncommitted(result); } - - public static boolean isOpenZen() { - boolean res = CommonParameter.getInstance().isFullNodeAllowShieldedTransactionArgs(); - if (res) { - INSTANCE = LibrustzcashWrapper.getInstance(); - } - return res; - } - } diff --git a/chainbase/src/main/java/org/tron/common/zksnark/JLibsodium.java b/chainbase/src/main/java/org/tron/common/zksnark/JLibsodium.java index 0159ba0bf6b..0713d74b7bd 100644 --- a/chainbase/src/main/java/org/tron/common/zksnark/JLibsodium.java +++ b/chainbase/src/main/java/org/tron/common/zksnark/JLibsodium.java @@ -12,37 +12,25 @@ public class JLibsodium { public static final int CRYPTO_GENERICHASH_BLAKE2B_PERSONALBYTES = 16; public static final int CRYPTO_AEAD_CHACHA20POLY1305_IETF_NPUBBYTES = 12; - private static Libsodium INSTANCE; + private static Libsodium INSTANCE = LibsodiumWrapper.getInstance(); public static int cryptoGenerichashBlake2bInitSaltPersonal(Blake2bInitSaltPersonalParams params) { - if (!isOpenZen()) { - return 0; - } return INSTANCE .cryptoGenerichashBlake2BInitSaltPersonal(params.getState(), params.getKey(), params.getKeyLen(), params.getOutLen(), params.getSalt(), params.getPersonal()); } public static int cryptoGenerichashBlake2bUpdate(Blake2bUpdateParams params) { - if (!isOpenZen()) { - return 0; - } return INSTANCE .cryptoGenerichashBlake2BUpdate(params.getState(), params.getIn(), params.getInLen()); } public static int cryptoGenerichashBlake2bFinal(Blake2bFinalParams params) { - if (!isOpenZen()) { - return 0; - } return INSTANCE.cryptoGenerichashBlake2BFinal(params.getState(), params.getOut(), params.getOutLen()); } public static int cryptoGenerichashBlack2bSaltPersonal(Black2bSaltPersonalParams params) { - if (!isOpenZen()) { - return 0; - } return INSTANCE.cryptoGenerichashBlake2BSaltPersonal(params.getOut(), params.getOutLen(), params.getIn(), params.getInLen(), params.getKey(), params.getKeyLen(), params.getSalt(), @@ -51,9 +39,6 @@ public static int cryptoGenerichashBlack2bSaltPersonal(Black2bSaltPersonalParams public static int cryptoAeadChacha20poly1305IetfDecrypt( Chacha20poly1305IetfDecryptParams params) { - if (!isOpenZen()) { - return 0; - } return INSTANCE .cryptoAeadChacha20Poly1305IetfDecrypt(params.getM(), params.getMLenP(), params.getNSec(), @@ -63,9 +48,6 @@ public static int cryptoAeadChacha20poly1305IetfDecrypt( public static int cryptoAeadChacha20Poly1305IetfEncrypt( Chacha20Poly1305IetfEncryptParams params) { - if (!isOpenZen()) { - return 0; - } return INSTANCE .cryptoAeadChacha20Poly1305IetfEncrypt(params.getC(), params.getCLenP(), params.getM(), params.getMLen(), params.getAd(), params.getAdLen(), @@ -73,25 +55,10 @@ public static int cryptoAeadChacha20Poly1305IetfEncrypt( } public static long initState() { - if (!isOpenZen()) { - return 0; - } return INSTANCE.cryptoGenerichashBlake2BStateInit(); } public static void freeState(long state) { - if (!isOpenZen()) { - return; - } INSTANCE.cryptoGenerichashBlake2BStateFree(state); } - - private static boolean isOpenZen() { - boolean res = CommonParameter.getInstance() - .isFullNodeAllowShieldedTransactionArgs(); - if (res) { - INSTANCE = LibsodiumWrapper.getInstance(); - } - return res; - } } diff --git a/chainbase/src/main/java/org/tron/core/capsule/TransactionCapsule.java b/chainbase/src/main/java/org/tron/core/capsule/TransactionCapsule.java index 95f436b19f0..b11c6b1e0a4 100755 --- a/chainbase/src/main/java/org/tron/core/capsule/TransactionCapsule.java +++ b/chainbase/src/main/java/org/tron/core/capsule/TransactionCapsule.java @@ -41,6 +41,7 @@ import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang3.ArrayUtils; import org.tron.common.crypto.ECKey.ECDSASignature; +import org.tron.common.crypto.Rsv; import org.tron.common.crypto.SignInterface; import org.tron.common.crypto.SignUtils; import org.tron.common.es.ExecutorServiceManager; @@ -456,14 +457,8 @@ public static long getCallValue(Transaction.Contract contract) { } public static String getBase64FromByteString(ByteString sign) { - byte[] r = sign.substring(0, 32).toByteArray(); - byte[] s = sign.substring(32, 64).toByteArray(); - byte v = sign.byteAt(64); - if (v < 27) { - v += 27; //revId -> v - } - ECDSASignature signature = ECDSASignature.fromComponents(r, s, v); - return signature.toBase64(); + Rsv rsv = Rsv.fromSignature(sign.toByteArray()); + return ECDSASignature.fromComponents(rsv.getR(), rsv.getS(), rsv.getV()).toBase64(); } public static boolean validateSignature(Transaction transaction, diff --git a/chainbase/src/main/java/org/tron/core/capsule/utils/MarketUtils.java b/chainbase/src/main/java/org/tron/core/capsule/utils/MarketUtils.java index d711ac0d63b..e98bba9db08 100644 --- a/chainbase/src/main/java/org/tron/core/capsule/utils/MarketUtils.java +++ b/chainbase/src/main/java/org/tron/core/capsule/utils/MarketUtils.java @@ -25,6 +25,7 @@ import org.tron.common.crypto.Hash; import org.tron.common.utils.ByteArray; import org.tron.common.utils.ByteUtil; +import org.tron.common.utils.MarketComparator; import org.tron.core.capsule.AccountCapsule; import org.tron.core.capsule.MarketAccountOrderCapsule; import org.tron.core.capsule.MarketOrderCapsule; @@ -227,29 +228,6 @@ public static byte[] createPairKey(byte[] sellTokenId, byte[] buyTokenId) { return result; } - /** - * Note: the params should be the same token pair, or you should change the order. - * All the quantity should be bigger than 0. - * */ - public static int comparePrice(long price1SellQuantity, long price1BuyQuantity, - long price2SellQuantity, long price2BuyQuantity) { - try { - return Long.compare(multiplyExact(price1BuyQuantity, price2SellQuantity, true), - multiplyExact(price2BuyQuantity, price1SellQuantity, true)); - - } catch (ArithmeticException ex) { - // do nothing here, because we will use BigInteger to compute again - } - - BigInteger price1BuyQuantityBI = BigInteger.valueOf(price1BuyQuantity); - BigInteger price1SellQuantityBI = BigInteger.valueOf(price1SellQuantity); - BigInteger price2BuyQuantityBI = BigInteger.valueOf(price2BuyQuantity); - BigInteger price2SellQuantityBI = BigInteger.valueOf(price2SellQuantity); - - return price1BuyQuantityBI.multiply(price2SellQuantityBI) - .compareTo(price2BuyQuantityBI.multiply(price1SellQuantityBI)); - } - /** * if takerPrice >= makerPrice, return True * note: here are two different token pairs @@ -265,7 +243,8 @@ public static boolean priceMatch(MarketPrice takerPrice, MarketPrice makerPrice) // ==> Price_TRX * sellQuantity_taker/buyQuantity_taker >= Price_TRX * buyQuantity_maker/sellQuantity_maker // ==> sellQuantity_taker * sellQuantity_maker > buyQuantity_taker * buyQuantity_maker - return comparePrice(takerPrice.getBuyTokenQuantity(), takerPrice.getSellTokenQuantity(), + return MarketComparator.comparePrice(takerPrice.getBuyTokenQuantity(), + takerPrice.getSellTokenQuantity(), makerPrice.getSellTokenQuantity(), makerPrice.getBuyTokenQuantity()) >= 0; } @@ -316,57 +295,7 @@ public static void returnSellTokenRemain(MarketOrderCapsule orderCapsule, } public static int comparePriceKey(byte[] o1, byte[] o2) { - //compare pair - byte[] pair1 = new byte[TOKEN_ID_LENGTH * 2]; - byte[] pair2 = new byte[TOKEN_ID_LENGTH * 2]; - - System.arraycopy(o1, 0, pair1, 0, TOKEN_ID_LENGTH * 2); - System.arraycopy(o2, 0, pair2, 0, TOKEN_ID_LENGTH * 2); - - int pairResult = org.bouncycastle.util.Arrays.compareUnsigned(pair1, pair2); - if (pairResult != 0) { - return pairResult; - } - - //compare price - byte[] getSellTokenQuantity1 = new byte[8]; - byte[] getBuyTokenQuantity1 = new byte[8]; - - byte[] getSellTokenQuantity2 = new byte[8]; - byte[] getBuyTokenQuantity2 = new byte[8]; - - int longByteNum = 8; - - System.arraycopy(o1, TOKEN_ID_LENGTH + TOKEN_ID_LENGTH, - getSellTokenQuantity1, 0, longByteNum); - System.arraycopy(o1, TOKEN_ID_LENGTH + TOKEN_ID_LENGTH + longByteNum, - getBuyTokenQuantity1, 0, longByteNum); - - System.arraycopy(o2, TOKEN_ID_LENGTH + TOKEN_ID_LENGTH, - getSellTokenQuantity2, 0, longByteNum); - System.arraycopy(o2, TOKEN_ID_LENGTH + TOKEN_ID_LENGTH + longByteNum, - getBuyTokenQuantity2, 0, longByteNum); - - long sellTokenQuantity1 = ByteArray.toLong(getSellTokenQuantity1); - long buyTokenQuantity1 = ByteArray.toLong(getBuyTokenQuantity1); - long sellTokenQuantity2 = ByteArray.toLong(getSellTokenQuantity2); - long buyTokenQuantity2 = ByteArray.toLong(getBuyTokenQuantity2); - - if ((sellTokenQuantity1 == 0 || buyTokenQuantity1 == 0) - && (sellTokenQuantity2 == 0 || buyTokenQuantity2 == 0)) { - return 0; - } - - if (sellTokenQuantity1 == 0 || buyTokenQuantity1 == 0) { - return -1; - } - - if (sellTokenQuantity2 == 0 || buyTokenQuantity2 == 0) { - return 1; - } - - return comparePrice(sellTokenQuantity1, buyTokenQuantity1, - sellTokenQuantity2, buyTokenQuantity2); + return MarketComparator.comparePriceKey(o1, o2); } diff --git a/chainbase/src/main/java/org/tron/core/db/TronDatabase.java b/chainbase/src/main/java/org/tron/core/db/TronDatabase.java index d791e189eb4..40762568c82 100644 --- a/chainbase/src/main/java/org/tron/core/db/TronDatabase.java +++ b/chainbase/src/main/java/org/tron/core/db/TronDatabase.java @@ -8,8 +8,6 @@ import javax.annotation.PostConstruct; import lombok.Getter; import lombok.extern.slf4j.Slf4j; -import org.iq80.leveldb.WriteOptions; -import org.rocksdb.DirectComparator; import org.springframework.beans.factory.annotation.Autowired; import org.tron.common.parameter.CommonParameter; import org.tron.common.storage.WriteOptionsWrapper; @@ -29,7 +27,7 @@ public abstract class TronDatabase implements ITronChainBase { protected DbSourceInter dbSource; @Getter private String dbName; - private WriteOptionsWrapper writeOptions = WriteOptionsWrapper.getInstance() + private final WriteOptionsWrapper writeOptions = WriteOptionsWrapper.getInstance() .sync(CommonParameter.getInstance().getStorage().isDbSync()); @Autowired @@ -40,22 +38,13 @@ protected TronDatabase(String dbName) { if ("LEVELDB".equals(CommonParameter.getInstance().getStorage() .getDbEngine().toUpperCase())) { - dbSource = - new LevelDbDataSourceImpl(StorageUtils.getOutputDirectoryByDbName(dbName), - dbName, - getOptionsByDbNameForLevelDB(dbName), - new WriteOptions().sync(CommonParameter.getInstance() - .getStorage().isDbSync())); + dbSource = new LevelDbDataSourceImpl(StorageUtils.getOutputDirectoryByDbName(dbName), dbName); } else if ("ROCKSDB".equals(CommonParameter.getInstance() .getStorage().getDbEngine().toUpperCase())) { String parentName = Paths.get(StorageUtils.getOutputDirectoryByDbName(dbName), CommonParameter.getInstance().getStorage().getDbDirectory()).toString(); - dbSource = - new RocksDbDataSourceImpl(parentName, dbName, CommonParameter.getInstance() - .getRocksDBCustomSettings(), getDirectComparator()); + dbSource = new RocksDbDataSourceImpl(parentName, dbName); } - - dbSource.initDB(); } @PostConstruct @@ -66,14 +55,6 @@ protected void init() { protected TronDatabase() { } - protected org.iq80.leveldb.Options getOptionsByDbNameForLevelDB(String dbName) { - return StorageUtils.getOptionsByDbName(dbName); - } - - protected DirectComparator getDirectComparator() { - return null; - } - public DbSourceInter getDbSource() { return dbSource; } @@ -96,6 +77,7 @@ public void reset() { public void close() { logger.info("******** Begin to close {}. ********", getName()); try { + writeOptions.close(); dbSource.closeDB(); } catch (Exception e) { logger.warn("Failed to close {}.", getName(), e); diff --git a/chainbase/src/main/java/org/tron/core/db/TronStoreWithRevoking.java b/chainbase/src/main/java/org/tron/core/db/TronStoreWithRevoking.java index 4b75ddee3a4..73b1b103d76 100644 --- a/chainbase/src/main/java/org/tron/core/db/TronStoreWithRevoking.java +++ b/chainbase/src/main/java/org/tron/core/db/TronStoreWithRevoking.java @@ -15,8 +15,6 @@ import javax.annotation.PostConstruct; import lombok.Getter; import lombok.extern.slf4j.Slf4j; -import org.iq80.leveldb.WriteOptions; -import org.rocksdb.DirectComparator; import org.springframework.beans.factory.annotation.Autowired; import org.tron.common.parameter.CommonParameter; import org.tron.common.storage.leveldb.LevelDbDataSourceImpl; @@ -58,33 +56,18 @@ protected TronStoreWithRevoking(String dbName) { String dbEngine = CommonParameter.getInstance().getStorage().getDbEngine(); if ("LEVELDB".equals(dbEngine.toUpperCase())) { this.db = new LevelDB( - new LevelDbDataSourceImpl(StorageUtils.getOutputDirectoryByDbName(dbName), - dbName, - getOptionsByDbNameForLevelDB(dbName), - new WriteOptions().sync(CommonParameter.getInstance() - .getStorage().isDbSync()))); + new LevelDbDataSourceImpl(StorageUtils.getOutputDirectoryByDbName(dbName), dbName)); } else if ("ROCKSDB".equals(dbEngine.toUpperCase())) { String parentPath = Paths .get(StorageUtils.getOutputDirectoryByDbName(dbName), CommonParameter .getInstance().getStorage().getDbDirectory()).toString(); - this.db = new RocksDB( - new RocksDbDataSourceImpl(parentPath, - dbName, CommonParameter.getInstance() - .getRocksDBCustomSettings(), getDirectComparator())); + this.db = new RocksDB(new RocksDbDataSourceImpl(parentPath, dbName)); } else { throw new RuntimeException(String.format("db engine %s is error", dbEngine)); } this.revokingDB = new Chainbase(new SnapshotRoot(this.db)); } - protected org.iq80.leveldb.Options getOptionsByDbNameForLevelDB(String dbName) { - return StorageUtils.getOptionsByDbName(dbName); - } - - protected DirectComparator getDirectComparator() { - return null; - } - protected TronStoreWithRevoking(DB db) { this.db = db; this.revokingDB = new Chainbase(new SnapshotRoot(db)); diff --git a/chainbase/src/main/java/org/tron/core/db/common/DbSourceInter.java b/chainbase/src/main/java/org/tron/core/db/common/DbSourceInter.java index 0823b7a7cf4..21c0a0dff2a 100755 --- a/chainbase/src/main/java/org/tron/core/db/common/DbSourceInter.java +++ b/chainbase/src/main/java/org/tron/core/db/common/DbSourceInter.java @@ -15,37 +15,82 @@ * You should have received a copy of the GNU Lesser General Public License * along with the ethereumJ library. If not, see . */ -package org.tron.core.db.common; -import org.tron.core.db2.common.WrappedByteArray; +package org.tron.core.db.common; +import com.google.common.annotations.VisibleForTesting; +import com.google.common.base.Strings; +import java.io.File; +import java.nio.file.Paths; import java.util.Map; import java.util.Set; - +import org.tron.common.utils.FileUtil; +import org.tron.common.utils.PropUtil; +import org.tron.core.db2.common.WrappedByteArray; +import org.tron.core.exception.TronError; public interface DbSourceInter extends BatchSourceInter, Iterable> { + String ENGINE_KEY = "ENGINE"; + String ENGINE_FILE = "engine.properties"; + String ROCKSDB = "ROCKSDB"; + String LEVELDB = "LEVELDB"; + String getDBName(); void setDBName(String name); - void initDB(); - boolean isAlive(); void closeDB(); void resetDb(); + @VisibleForTesting + @Deprecated Set allKeys() throws RuntimeException; + @VisibleForTesting + @Deprecated Set allValues() throws RuntimeException; + @VisibleForTesting + @Deprecated long getTotal() throws RuntimeException; void stat(); Map prefixQuery(byte[] key); + static void checkOrInitEngine(String expectedEngine, String dir, TronError.ErrCode errCode) { + String engineFile = Paths.get(dir, ENGINE_FILE).toString(); + File currentFile = new File(dir, "CURRENT"); + if (ROCKSDB.equals(expectedEngine) && currentFile.exists() + && !Paths.get(engineFile).toFile().exists()) { + // if the CURRENT file exists, but the engine.properties file does not exist, it is LevelDB + // 000003.log CURRENT LOCK MANIFEST-000002 + throw new TronError( + String.format("Cannot open %s database with %s engine.", LEVELDB, ROCKSDB), errCode); + } + if (FileUtil.createDirIfNotExists(dir)) { + if (!FileUtil.createFileIfNotExists(engineFile)) { + throw new TronError(String.format("Cannot create file: %s.", engineFile), errCode); + } + } else { + throw new TronError(String.format("Cannot create dir: %s.", dir), errCode); + } + String actualEngine = PropUtil.readProperty(engineFile, ENGINE_KEY); + // engine init + if (Strings.isNullOrEmpty(actualEngine) + && !PropUtil.writeProperty(engineFile, ENGINE_KEY, expectedEngine)) { + throw new TronError(String.format("Cannot write file: %s.", engineFile), errCode); + } + actualEngine = PropUtil.readProperty(engineFile, ENGINE_KEY); + if (!expectedEngine.equals(actualEngine)) { + throw new TronError(String.format( + "Cannot open %s database with %s engine.", + actualEngine, expectedEngine), errCode); + } + } } diff --git a/chainbase/src/main/java/org/tron/core/db/common/iterator/RockStoreIterator.java b/chainbase/src/main/java/org/tron/core/db/common/iterator/RockStoreIterator.java index 541f71348af..cf9a5ff1e22 100644 --- a/chainbase/src/main/java/org/tron/core/db/common/iterator/RockStoreIterator.java +++ b/chainbase/src/main/java/org/tron/core/db/common/iterator/RockStoreIterator.java @@ -5,6 +5,7 @@ import java.util.NoSuchElementException; import java.util.concurrent.atomic.AtomicBoolean; import lombok.extern.slf4j.Slf4j; +import org.rocksdb.ReadOptions; import org.rocksdb.RocksIterator; @@ -15,14 +16,17 @@ public final class RockStoreIterator implements DBIterator { private boolean first = true; private final AtomicBoolean close = new AtomicBoolean(false); + private final ReadOptions readOptions; - public RockStoreIterator(RocksIterator dbIterator) { + public RockStoreIterator(RocksIterator dbIterator, ReadOptions readOptions) { + this.readOptions = readOptions; this.dbIterator = dbIterator; } @Override public void close() throws IOException { if (close.compareAndSet(false, true)) { + readOptions.close(); dbIterator.close(); } } @@ -47,7 +51,7 @@ public boolean hasNext() { try { close(); } catch (Exception e1) { - logger.error(e.getMessage(), e); + logger.error(e1.getMessage(), e1); } } return hasNext; @@ -79,6 +83,11 @@ public byte[] setValue(byte[] value) { }; } + @Override + public void remove() { + throw new UnsupportedOperationException(); + } + @Override public void seek(byte[] key) { checkState(); diff --git a/chainbase/src/main/java/org/tron/core/db/common/iterator/StoreIterator.java b/chainbase/src/main/java/org/tron/core/db/common/iterator/StoreIterator.java index d771716a7e8..c2803f99637 100755 --- a/chainbase/src/main/java/org/tron/core/db/common/iterator/StoreIterator.java +++ b/chainbase/src/main/java/org/tron/core/db/common/iterator/StoreIterator.java @@ -46,6 +46,11 @@ public boolean hasNext() { } } catch (Exception e) { logger.error(e.getMessage(), e); + try { + close(); + } catch (Exception e1) { + logger.error(e1.getMessage(), e1); + } } return hasNext; diff --git a/chainbase/src/main/java/org/tron/core/db2/common/LevelDB.java b/chainbase/src/main/java/org/tron/core/db2/common/LevelDB.java index b5616f87b8a..5942bb7444c 100644 --- a/chainbase/src/main/java/org/tron/core/db2/common/LevelDB.java +++ b/chainbase/src/main/java/org/tron/core/db2/common/LevelDB.java @@ -13,7 +13,7 @@ public class LevelDB implements DB, Flusher { @Getter private LevelDbDataSourceImpl db; - private WriteOptionsWrapper writeOptions = WriteOptionsWrapper.getInstance() + private final WriteOptionsWrapper writeOptions = WriteOptionsWrapper.getInstance() .sync(CommonParameter.getInstance().getStorage().isDbSync()); public LevelDB(LevelDbDataSourceImpl db) { @@ -65,6 +65,7 @@ public void flush(Map batch) { @Override public void close() { + this.writeOptions.close(); db.closeDB(); } diff --git a/chainbase/src/main/java/org/tron/core/db2/common/RocksDB.java b/chainbase/src/main/java/org/tron/core/db2/common/RocksDB.java index 31970b185cc..1d67438eceb 100644 --- a/chainbase/src/main/java/org/tron/core/db2/common/RocksDB.java +++ b/chainbase/src/main/java/org/tron/core/db2/common/RocksDB.java @@ -14,7 +14,7 @@ public class RocksDB implements DB, Flusher { @Getter private RocksDbDataSourceImpl db; - private WriteOptionsWrapper optionsWrapper = WriteOptionsWrapper.getInstance() + private final WriteOptionsWrapper writeOptions = WriteOptionsWrapper.getInstance() .sync(CommonParameter.getInstance().getStorage().isDbSync()); public RocksDB(RocksDbDataSourceImpl db) { @@ -61,11 +61,12 @@ public void flush(Map batch) { Map rows = batch.entrySet().stream() .map(e -> Maps.immutableEntry(e.getKey().getBytes(), e.getValue().getBytes())) .collect(HashMap::new, (m, k) -> m.put(k.getKey(), k.getValue()), HashMap::putAll); - db.updateByBatch(rows, optionsWrapper); + db.updateByBatch(rows, writeOptions); } @Override public void close() { + writeOptions.close(); db.closeDB(); } diff --git a/chainbase/src/main/java/org/tron/core/db2/common/TxCacheDB.java b/chainbase/src/main/java/org/tron/core/db2/common/TxCacheDB.java index 9d2409685c9..31131de0866 100644 --- a/chainbase/src/main/java/org/tron/core/db2/common/TxCacheDB.java +++ b/chainbase/src/main/java/org/tron/core/db2/common/TxCacheDB.java @@ -31,7 +31,6 @@ import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang3.ArrayUtils; import org.bouncycastle.util.encoders.Hex; -import org.iq80.leveldb.WriteOptions; import org.tron.common.parameter.CommonParameter; import org.tron.common.prometheus.MetricKeys; import org.tron.common.prometheus.Metrics; @@ -105,19 +104,13 @@ public TxCacheDB(String name, RecentTransactionStore recentTransactionStore, String dbEngine = CommonParameter.getInstance().getStorage().getDbEngine(); if ("LEVELDB".equals(dbEngine.toUpperCase())) { this.persistentStore = new LevelDB( - new LevelDbDataSourceImpl(StorageUtils.getOutputDirectoryByDbName(name), - name, StorageUtils.getOptionsByDbName(name), - new WriteOptions().sync(CommonParameter.getInstance() - .getStorage().isDbSync()))); + new LevelDbDataSourceImpl(StorageUtils.getOutputDirectoryByDbName(name), name)); } else if ("ROCKSDB".equals(dbEngine.toUpperCase())) { String parentPath = Paths .get(StorageUtils.getOutputDirectoryByDbName(name), CommonParameter .getInstance().getStorage().getDbDirectory()).toString(); - this.persistentStore = new RocksDB( - new RocksDbDataSourceImpl(parentPath, - name, CommonParameter.getInstance() - .getRocksDBCustomSettings())); + this.persistentStore = new RocksDB(new RocksDbDataSourceImpl(parentPath, name)); } else { throw new RuntimeException(String.format("db type: %s is not supported", dbEngine)); } diff --git a/chainbase/src/main/java/org/tron/core/db2/core/SnapshotManager.java b/chainbase/src/main/java/org/tron/core/db2/core/SnapshotManager.java index eb27141a82c..e20490d93c0 100644 --- a/chainbase/src/main/java/org/tron/core/db2/core/SnapshotManager.java +++ b/chainbase/src/main/java/org/tron/core/db2/core/SnapshotManager.java @@ -29,7 +29,6 @@ import org.tron.common.error.TronDBException; import org.tron.common.es.ExecutorServiceManager; import org.tron.common.parameter.CommonParameter; -import org.tron.common.storage.WriteOptionsWrapper; import org.tron.common.utils.FileUtil; import org.tron.common.utils.StorageUtils; import org.tron.core.db.RevokingDatabase; @@ -357,7 +356,6 @@ public void flush() { public void createCheckpoint() { TronDatabase checkPointStore = null; - boolean syncFlag; try { Map batch = new HashMap<>(); for (Chainbase db : dbs) { @@ -389,16 +387,13 @@ public void createCheckpoint() { if (isV2Open()) { String dbName = String.valueOf(System.currentTimeMillis()); checkPointStore = getCheckpointDB(dbName); - syncFlag = CommonParameter.getInstance().getStorage().isCheckpointSync(); } else { checkPointStore = checkTmpStore; - syncFlag = CommonParameter.getInstance().getStorage().isDbSync(); } - checkPointStore.getDbSource().updateByBatch(batch.entrySet().stream() + checkPointStore.updateByBatch(batch.entrySet().stream() .map(e -> Maps.immutableEntry(e.getKey().getBytes(), e.getValue().getBytes())) - .collect(HashMap::new, (m, k) -> m.put(k.getKey(), k.getValue()), HashMap::putAll), - WriteOptionsWrapper.getInstance().sync(syncFlag)); + .collect(HashMap::new, (m, k) -> m.put(k.getKey(), k.getValue()), HashMap::putAll)); } catch (Exception e) { throw new TronDBException(e); @@ -427,6 +422,10 @@ public List getCheckpointList() { } private void deleteCheckpoint() { + if(checkTmpStore == null) { + // only occurs in mock test. TODO fix test + return; + } try { Map hmap = new HashMap<>(); for (Map.Entry e : checkTmpStore.getDbSource()) { diff --git a/chainbase/src/main/java/org/tron/core/store/AccountStore.java b/chainbase/src/main/java/org/tron/core/store/AccountStore.java index 4d39049ee79..5aec5958729 100644 --- a/chainbase/src/main/java/org/tron/core/store/AccountStore.java +++ b/chainbase/src/main/java/org/tron/core/store/AccountStore.java @@ -12,6 +12,7 @@ import org.tron.core.capsule.BlockCapsule; import org.tron.core.db.TronStoreWithRevoking; import org.tron.core.db.accountstate.AccountStateCallBackUtils; +import org.tron.core.exception.TronError; import org.tron.protos.contract.BalanceContract.TransactionBalanceTrace; import org.tron.protos.contract.BalanceContract.TransactionBalanceTrace.Operation; @@ -23,6 +24,8 @@ @Component public class AccountStore extends TronStoreWithRevoking { + private static String ACCOUNT_BLACKHOLE = "Blackhole"; + private static Map assertsAddress = new HashMap<>(); // key = name , value = address @Autowired @@ -50,6 +53,9 @@ public static void setAccount(com.typesafe.config.Config config) { byte[] address = Commons.decodeFromBase58Check(obj.get("address").unwrapped().toString()); assertsAddress.put(accountName, address); } + if (assertsAddress.get(ACCOUNT_BLACKHOLE) == null) { + throw new TronError("Account[Blackhole] is not configured.", TronError.ErrCode.GENESIS_BLOCK_INIT); + } } @Override @@ -109,12 +115,12 @@ public AccountCapsule getSun() { * Min TRX account. */ public AccountCapsule getBlackhole() { - return getUnchecked(assertsAddress.get("Blackhole")); + return getUnchecked(assertsAddress.get(ACCOUNT_BLACKHOLE)); } public byte[] getBlackholeAddress() { - return assertsAddress.get("Blackhole"); + return assertsAddress.get(ACCOUNT_BLACKHOLE); } /** diff --git a/chainbase/src/main/java/org/tron/core/store/CheckPointV2Store.java b/chainbase/src/main/java/org/tron/core/store/CheckPointV2Store.java index 2f952e6b82a..f027bd02664 100644 --- a/chainbase/src/main/java/org/tron/core/store/CheckPointV2Store.java +++ b/chainbase/src/main/java/org/tron/core/store/CheckPointV2Store.java @@ -1,18 +1,23 @@ package org.tron.core.store; import com.google.protobuf.InvalidProtocolBufferException; +import java.util.Map; +import java.util.Spliterator; +import java.util.function.Consumer; import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Autowired; +import org.tron.common.parameter.CommonParameter; +import org.tron.common.storage.WriteOptionsWrapper; import org.tron.core.db.TronDatabase; import org.tron.core.exception.BadItemException; import org.tron.core.exception.ItemNotFoundException; -import java.util.Spliterator; -import java.util.function.Consumer; - @Slf4j(topic = "DB") public class CheckPointV2Store extends TronDatabase { + private final WriteOptionsWrapper writeOptions = WriteOptionsWrapper.getInstance() + .sync(CommonParameter.getInstance().getStorage().isCheckpointSync()); + @Autowired public CheckPointV2Store(String dbPath) { super(dbPath); @@ -52,6 +57,11 @@ public Spliterator spliterator() { protected void init() { } + @Override + public void updateByBatch(Map rows) { + this.dbSource.updateByBatch(rows, writeOptions); + } + /** * close the database. */ @@ -59,6 +69,7 @@ protected void init() { public void close() { logger.debug("******** Begin to close {}. ********", getName()); try { + writeOptions.close(); dbSource.closeDB(); } catch (Exception e) { logger.warn("Failed to close {}.", getName(), e); diff --git a/chainbase/src/main/java/org/tron/core/store/DynamicPropertiesStore.java b/chainbase/src/main/java/org/tron/core/store/DynamicPropertiesStore.java index 91b0cff68a0..89c5ba18e59 100644 --- a/chainbase/src/main/java/org/tron/core/store/DynamicPropertiesStore.java +++ b/chainbase/src/main/java/org/tron/core/store/DynamicPropertiesStore.java @@ -1,6 +1,8 @@ package org.tron.core.store; import static org.tron.common.math.Maths.max; +import static org.tron.core.Constant.MAX_PROPOSAL_EXPIRE_TIME; +import static org.tron.core.Constant.MIN_PROPOSAL_EXPIRE_TIME; import static org.tron.core.config.Parameter.ChainConstant.BLOCK_PRODUCED_INTERVAL; import static org.tron.core.config.Parameter.ChainConstant.DELEGATE_PERIOD; @@ -231,6 +233,10 @@ public class DynamicPropertiesStore extends TronStoreWithRevoking private static final byte[] ALLOW_TVM_CANCUN = "ALLOW_TVM_CANCUN".getBytes(); private static final byte[] ALLOW_TVM_BLOB = "ALLOW_TVM_BLOB".getBytes(); + private static final byte[] PROPOSAL_EXPIRE_TIME = "PROPOSAL_EXPIRE_TIME".getBytes(); + + private static final byte[] ALLOW_TVM_SELFDESTRUCT_RESTRICTION = + "ALLOW_TVM_SELFDESTRUCT_RESTRICTION".getBytes(); @Autowired private DynamicPropertiesStore(@Value("properties") String dbName) { @@ -2946,6 +2952,34 @@ public long getAllowTvmBlob() { .orElse(CommonParameter.getInstance().getAllowTvmBlob()); } + + public long getAllowTvmSelfdestructRestriction() { + return Optional.ofNullable(getUnchecked(ALLOW_TVM_SELFDESTRUCT_RESTRICTION)) + .map(BytesCapsule::getData) + .map(ByteArray::toLong) + .orElse(0L); + } + + public void saveAllowTvmSelfdestructRestriction(long value) { + this.put(ALLOW_TVM_SELFDESTRUCT_RESTRICTION, new BytesCapsule(ByteArray.fromLong(value))); + } + + public boolean allowTvmSelfdestructRestriction() { + return getAllowTvmSelfdestructRestriction() == 1L; + } + + public void saveProposalExpireTime(long proposalExpireTime) { + this.put(PROPOSAL_EXPIRE_TIME, new BytesCapsule(ByteArray.fromLong(proposalExpireTime))); + } + + public long getProposalExpireTime() { + return Optional.ofNullable(getUnchecked(PROPOSAL_EXPIRE_TIME)) + .map(BytesCapsule::getData) + .map(ByteArray::toLong) + .filter(time -> time > MIN_PROPOSAL_EXPIRE_TIME && time < MAX_PROPOSAL_EXPIRE_TIME) + .orElse(CommonParameter.getInstance().getProposalExpireTime()); + } + private static class DynamicResourceProperties { private static final byte[] ONE_DAY_NET_LIMIT = "ONE_DAY_NET_LIMIT".getBytes(); diff --git a/chainbase/src/main/java/org/tron/core/store/MarketPairPriceToOrderStore.java b/chainbase/src/main/java/org/tron/core/store/MarketPairPriceToOrderStore.java index 605952328ed..391fb4249c8 100644 --- a/chainbase/src/main/java/org/tron/core/store/MarketPairPriceToOrderStore.java +++ b/chainbase/src/main/java/org/tron/core/store/MarketPairPriceToOrderStore.java @@ -3,16 +3,11 @@ import java.util.ArrayList; import java.util.Collections; import java.util.List; -import org.iq80.leveldb.Options; -import org.rocksdb.ComparatorOptions; -import org.rocksdb.DirectComparator; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Value; import org.springframework.stereotype.Component; import org.tron.common.utils.ByteUtil; -import org.tron.common.utils.MarketOrderPriceComparatorForLevelDB; -import org.tron.common.utils.MarketOrderPriceComparatorForRockDB; -import org.tron.common.utils.StorageUtils; +import org.tron.core.Constant; import org.tron.core.capsule.MarketOrderIdListCapsule; import org.tron.core.capsule.utils.MarketUtils; import org.tron.core.db.TronStoreWithRevoking; @@ -22,24 +17,10 @@ public class MarketPairPriceToOrderStore extends TronStoreWithRevoking { @Autowired - protected MarketPairPriceToOrderStore(@Value("market_pair_price_to_order") String dbName) { + protected MarketPairPriceToOrderStore(@Value(Constant.MARKET_PAIR_PRICE_TO_ORDER) String dbName) { super(dbName); } - @Override - protected Options getOptionsByDbNameForLevelDB(String dbName) { - Options options = StorageUtils.getOptionsByDbName(dbName); - options.comparator(new MarketOrderPriceComparatorForLevelDB()); - return options; - } - - //todo: to test later - @Override - protected DirectComparator getDirectComparator() { - ComparatorOptions comparatorOptions = new ComparatorOptions(); - return new MarketOrderPriceComparatorForRockDB(comparatorOptions); - } - @Override public MarketOrderIdListCapsule get(byte[] key) throws ItemNotFoundException { byte[] value = revokingDB.get(key); diff --git a/chainbase/src/main/java/org/tron/core/store/WitnessStore.java b/chainbase/src/main/java/org/tron/core/store/WitnessStore.java index 9f444d3333d..9c30df195af 100644 --- a/chainbase/src/main/java/org/tron/core/store/WitnessStore.java +++ b/chainbase/src/main/java/org/tron/core/store/WitnessStore.java @@ -55,7 +55,7 @@ public List getWitnessStandby(boolean isSortOpt) { return ret; } - public void sortWitnesses(List witnesses, boolean isSortOpt) { + public static void sortWitnesses(List witnesses, boolean isSortOpt) { witnesses.sort(Comparator.comparingLong(WitnessCapsule::getVoteCount).reversed() .thenComparing(isSortOpt ? Comparator.comparing(WitnessCapsule::createReadableString).reversed() diff --git a/common/build.gradle b/common/build.gradle index c6ce8cf44f9..8ea91ecd5b1 100644 --- a/common/build.gradle +++ b/common/build.gradle @@ -6,45 +6,23 @@ version '1.0.0' sourceCompatibility = 1.8 -// Dependency versions -// --------------------------------------- -def leveldbVersion = "1.8" -// -------------------------------------- - -static def isWindows() { - return org.gradle.internal.os.OperatingSystem.current().isWindows() -} - -if (isWindows()) { - ext { - leveldbGroup = "org.ethereum" - leveldbName = "leveldbjni-all" - leveldbVersion = "1.18.3" - } -} else { - ext { - leveldbGroup = "org.fusesource.leveldbjni" - leveldbName = "leveldbjni-all" - leveldbVersion = "1.8" - } -} dependencies { - api group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.13.4.2' // https://github.com/FasterXML/jackson-databind/issues/3627 - api "com.cedarsoftware:java-util:1.8.0" + api group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.18.3' // https://github.com/FasterXML/jackson-databind/issues/3627 + api "com.cedarsoftware:java-util:3.2.0" api group: 'org.apache.httpcomponents', name: 'httpasyncclient', version: '4.1.1' api group: 'commons-codec', name: 'commons-codec', version: '1.11' api group: 'com.beust', name: 'jcommander', version: '1.78' api group: 'com.typesafe', name: 'config', version: '1.3.2' - api group: leveldbGroup, name: leveldbName, version: leveldbVersion - api group: 'org.rocksdb', name: 'rocksdbjni', version: '5.15.10' api group: 'io.prometheus', name: 'simpleclient', version: '0.15.0' api group: 'io.prometheus', name: 'simpleclient_httpserver', version: '0.15.0' api group: 'io.prometheus', name: 'simpleclient_hotspot', version: '0.15.0' - api 'org.aspectj:aspectjrt:1.8.13' - api 'org.aspectj:aspectjweaver:1.8.13' - api 'org.aspectj:aspectjtools:1.8.13' - api group: 'io.github.tronprotocol', name: 'libp2p', version: '2.2.5',{ + // https://openjdk.org/jeps/396, JEP 396: Strongly Encapsulate JDK Internals by Default + // https://eclipse.dev/aspectj/doc/latest/release/JavaVersionCompatibility.html + api 'org.aspectj:aspectjrt:1.9.8' + api 'org.aspectj:aspectjweaver:1.9.8' + api 'org.aspectj:aspectjtools:1.9.8' + api group: 'io.github.tronprotocol', name: 'libp2p', version: '2.2.7',{ exclude group: 'io.grpc', module: 'grpc-context' exclude group: 'io.grpc', module: 'grpc-core' exclude group: 'io.grpc', module: 'grpc-netty' @@ -62,6 +40,7 @@ dependencies { exclude group: 'org.bouncycastle', module: 'bcutil-jdk18on' } api project(":protocol") + api project(":platform") } jacocoTestReport { diff --git a/common/src/main/java/org/tron/common/exit/ExitManager.java b/common/src/main/java/org/tron/common/exit/ExitManager.java index d80b7838c08..1a1cc9ed3fc 100644 --- a/common/src/main/java/org/tron/common/exit/ExitManager.java +++ b/common/src/main/java/org/tron/common/exit/ExitManager.java @@ -46,7 +46,7 @@ public static Optional findTronError(Throwable e) { public static void logAndExit(TronError exit) { final int code = exit.getErrCode().getCode(); - logger.error("Shutting down with code: {}.", exit.getErrCode(), exit); + logger.error("Shutting down with code: {}, reason: {}", exit.getErrCode(), exit.getMessage()); Thread exitThread = exitThreadFactory.newThread(() -> System.exit(code)); exitThread.start(); } diff --git a/common/src/main/java/org/tron/common/log/LogService.java b/common/src/main/java/org/tron/common/log/LogService.java index 801e13f54f7..bce52001e92 100644 --- a/common/src/main/java/org/tron/common/log/LogService.java +++ b/common/src/main/java/org/tron/common/log/LogService.java @@ -2,6 +2,7 @@ import ch.qos.logback.classic.LoggerContext; import ch.qos.logback.classic.joran.JoranConfigurator; +import ch.qos.logback.core.util.StatusPrinter; import java.io.File; import org.slf4j.LoggerFactory; import org.tron.core.exception.TronError; @@ -9,18 +10,20 @@ public class LogService { public static void load(String path) { + LoggerContext lc = (LoggerContext) LoggerFactory.getILoggerFactory(); try { File file = new File(path); if (!file.exists() || !file.isFile() || !file.canRead()) { return; } - LoggerContext lc = (LoggerContext) LoggerFactory.getILoggerFactory(); JoranConfigurator configurator = new JoranConfigurator(); configurator.setContext(lc); lc.reset(); configurator.doConfigure(file); } catch (Exception e) { throw new TronError(e, TronError.ErrCode.LOG_LOAD); + } finally { + StatusPrinter.printInCaseOfErrorsOrWarnings(lc); } } } diff --git a/common/src/main/java/org/tron/common/logsfilter/FilterQuery.java b/common/src/main/java/org/tron/common/logsfilter/FilterQuery.java index b2d0fc428e4..2ae50370129 100644 --- a/common/src/main/java/org/tron/common/logsfilter/FilterQuery.java +++ b/common/src/main/java/org/tron/common/logsfilter/FilterQuery.java @@ -31,12 +31,7 @@ public static long parseFromBlockNumber(String blockNum) { if (StringUtils.isEmpty(blockNum) || FilterQuery.EARLIEST.equalsIgnoreCase(blockNum)) { number = FilterQuery.EARLIEST_BLOCK_NUM; } else { - try { - number = Long.parseLong(blockNum); - } catch (Exception e) { - logger.error("invalid filter: fromBlockNumber: {}", blockNum); - throw e; - } + number = Long.parseLong(blockNum); } return number; } @@ -46,12 +41,7 @@ public static long parseToBlockNumber(String blockNum) { if (StringUtils.isEmpty(blockNum) || FilterQuery.LATEST.equalsIgnoreCase(blockNum)) { number = FilterQuery.LATEST_BLOCK_NUM; } else { - try { - number = Long.parseLong(blockNum); - } catch (Exception e) { - logger.error("invalid filter: toBlockNumber: {}", blockNum); - throw e; - } + number = Long.parseLong(blockNum); } return number; } diff --git a/common/src/main/java/org/tron/common/parameter/CommonParameter.java b/common/src/main/java/org/tron/common/parameter/CommonParameter.java index 45893970fb0..0583962f266 100644 --- a/common/src/main/java/org/tron/common/parameter/CommonParameter.java +++ b/common/src/main/java/org/tron/common/parameter/CommonParameter.java @@ -67,7 +67,7 @@ public class CommonParameter { public boolean debug = false; @Getter @Setter - @Parameter(names = {"--min-time-ratio"}, description = "Maximum CPU tolerance when executing " + @Parameter(names = {"--min-time-ratio"}, description = "Minimum CPU tolerance when executing " + "timeout transactions while synchronizing blocks. (default: 0.0)") public double minTimeRatio = 0.0; @Getter @@ -205,7 +205,15 @@ public class CommonParameter { //If you are running a solidity node for java tron, this flag is set to true @Getter @Setter + @Parameter(names = {"--solidity"}, description = "running a solidity node for java tron") public boolean solidityNode = false; + + //If you are running KeystoreFactory, this flag is set to true + @Getter + @Setter + @Parameter(names = {"--keystore-factory"}, description = "running KeystoreFactory") + public boolean keystoreFactory = false; + @Getter @Setter public int rpcPort; @@ -241,6 +249,15 @@ public class CommonParameter { @Getter @Setter public int flowControlWindow; + // the positive limit of RST_STREAM frames per connection per period for grpc, + // 0 or Integer.MAX_VALUE for unlimited, by default there is no limit. + @Getter + @Setter + public int rpcMaxRstStream; + // the positive number of seconds per period for grpc + @Getter + @Setter + public int rpcSecondsPerWindow; @Getter @Setter public long maxConnectionIdleInMillis; @@ -383,7 +400,7 @@ public class CommonParameter { // full node used this parameter to close shielded transaction @Getter @Setter - public boolean fullNodeAllowShieldedTransactionArgs; + public boolean allowShieldedTransactionApi; @Getter @Setter public long blockNumForEnergyLimit; @@ -429,6 +446,15 @@ public class CommonParameter { @Getter public int rateLimiterGlobalApiQps; @Getter + @Setter + public double rateLimiterSyncBlockChain; + @Getter + @Setter + public double rateLimiterFetchInvData; + @Getter + @Setter + public double rateLimiterDisconnect; + @Getter public DbBackupConfig dbBackupConfig; @Getter public RocksDbSettings rocksDBCustomSettings; @@ -501,6 +527,9 @@ public class CommonParameter { @Getter @Setter public int jsonRpcMaxSubTopics = 1000; + @Getter + @Setter + public int jsonRpcMaxBlockFilterNum = 50000; @Getter @Setter diff --git a/common/src/main/java/org/tron/common/setting/RocksDbSettings.java b/common/src/main/java/org/tron/common/setting/RocksDbSettings.java index 7436150cae2..d5df5e261b5 100644 --- a/common/src/main/java/org/tron/common/setting/RocksDbSettings.java +++ b/common/src/main/java/org/tron/common/setting/RocksDbSettings.java @@ -1,16 +1,26 @@ package org.tron.common.setting; +import static org.tron.core.Constant.ROCKSDB; + +import java.util.Arrays; import lombok.Getter; -import lombok.Setter; import lombok.extern.slf4j.Slf4j; +import org.rocksdb.BlockBasedTableConfig; +import org.rocksdb.BloomFilter; +import org.rocksdb.ComparatorOptions; +import org.rocksdb.InfoLogLevel; import org.rocksdb.LRUCache; +import org.rocksdb.Logger; +import org.rocksdb.Options; import org.rocksdb.RocksDB; +import org.rocksdb.Statistics; +import org.slf4j.LoggerFactory; +import org.tron.common.utils.MarketOrderPriceComparatorForRocksDB; +import org.tron.core.Constant; @Slf4j public class RocksDbSettings { - @Setter - @Getter private static RocksDbSettings rocksDbSettings; @Getter @@ -40,6 +50,17 @@ public class RocksDbSettings { private static final LRUCache cache = new LRUCache(1 * 1024 * 1024 * 1024L); + private static final String[] CI_ENVIRONMENT_VARIABLES = { + "CI", + "JENKINS_URL", + "TRAVIS", + "CIRCLECI", + "GITHUB_ACTIONS", + "GITLAB_CI" + }; + + private static final org.slf4j.Logger rocksDbLogger = LoggerFactory.getLogger(ROCKSDB); + private RocksDbSettings() { } @@ -60,9 +81,9 @@ public static RocksDbSettings initCustomSettings(int levelNumber, int compactThr int blockSize, long maxBytesForLevelBase, double maxBytesForLevelMultiplier, int level0FileNumCompactionTrigger, long targetFileSizeBase, - int targetFileSizeMultiplier) { + int targetFileSizeMultiplier, int maxOpenFiles) { rocksDbSettings = new RocksDbSettings() - .withMaxOpenFiles(5000) + .withMaxOpenFiles(maxOpenFiles) .withEnableStatistics(false) .withLevelNumber(levelNumber) .withCompactThreads(compactThreads) @@ -76,16 +97,17 @@ public static RocksDbSettings initCustomSettings(int levelNumber, int compactThr } public static void loggingSettings() { - logger.info(String.format( - "level number: %d, CompactThreads: %d, Blocksize: %d, maxBytesForLevelBase: %d," - + " withMaxBytesForLevelMultiplier: %f, level0FileNumCompactionTrigger: %d, " - + "withTargetFileSizeBase: %d, withTargetFileSizeMultiplier: %d", + logger.info( + "level number: {}, CompactThreads: {}, Blocksize:{}, maxBytesForLevelBase: {}," + + " withMaxBytesForLevelMultiplier: {}, level0FileNumCompactionTrigger: {}, " + + "withTargetFileSizeBase: {}, withTargetFileSizeMultiplier: {}, maxOpenFiles: {}", rocksDbSettings.getLevelNumber(), rocksDbSettings.getCompactThreads(), rocksDbSettings.getBlockSize(), rocksDbSettings.getMaxBytesForLevelBase(), rocksDbSettings.getMaxBytesForLevelMultiplier(), rocksDbSettings.getLevel0FileNumCompactionTrigger(), - rocksDbSettings.getTargetFileSizeBase(), rocksDbSettings.getTargetFileSizeMultiplier())); + rocksDbSettings.getTargetFileSizeBase(), rocksDbSettings.getTargetFileSizeMultiplier(), + rocksDbSettings.getMaxOpenFiles()); } public RocksDbSettings withMaxOpenFiles(int maxOpenFiles) { @@ -140,4 +162,85 @@ public RocksDbSettings withTargetFileSizeMultiplier(int targetFileSizeMultiplier public static LRUCache getCache() { return cache; } + + /** + * Creates a new RocksDB Options. + * + *

CRITICAL: Must be closed after use to prevent native memory leaks. + * Use try-with-resources. + * + *

{@code
+   * try (Options options = getOptionsByDbName(dbName)) {
+   *     // do something
+   * }
+   * }
+ * + * @param dbName db name + * @return a new Options instance that must be closed + */ + public static Options getOptionsByDbName(String dbName) { + RocksDbSettings settings = getSettings(); + + Options options = new Options(); + + options.setLogger(new Logger(options) { + @Override + protected void log(InfoLogLevel infoLogLevel, String logMsg) { + rocksDbLogger.info("{} {}", dbName, logMsg); + } + }); + // most of these options are suggested by https://github.com/facebook/rocksdb/wiki/Set-Up-Options + + // general options + if (settings.isEnableStatistics()) { + options.setStatistics(new Statistics()); + options.setStatsDumpPeriodSec(60); + } + options.setCreateIfMissing(true); + options.setIncreaseParallelism(1); + options.setLevelCompactionDynamicLevelBytes(true); + options.setMaxOpenFiles(settings.getMaxOpenFiles()); + + // general options supported user config + options.setNumLevels(settings.getLevelNumber()); + options.setMaxBytesForLevelMultiplier(settings.getMaxBytesForLevelMultiplier()); + options.setMaxBytesForLevelBase(settings.getMaxBytesForLevelBase()); + options.setMaxBackgroundCompactions(settings.getCompactThreads()); + options.setLevel0FileNumCompactionTrigger(settings.getLevel0FileNumCompactionTrigger()); + options.setTargetFileSizeMultiplier(settings.getTargetFileSizeMultiplier()); + options.setTargetFileSizeBase(settings.getTargetFileSizeBase()); + + // table options + final BlockBasedTableConfig tableCfg; + options.setTableFormatConfig(tableCfg = new BlockBasedTableConfig()); + tableCfg.setBlockSize(settings.getBlockSize()); + tableCfg.setBlockCache(RocksDbSettings.getCache()); + tableCfg.setCacheIndexAndFilterBlocks(true); + tableCfg.setPinL0FilterAndIndexBlocksInCache(true); + tableCfg.setFilter(new BloomFilter(10, false)); + if (Constant.MARKET_PAIR_PRICE_TO_ORDER.equals(dbName)) { + ComparatorOptions comparatorOptions = new ComparatorOptions(); + options.setComparator(new MarketOrderPriceComparatorForRocksDB(comparatorOptions)); + } + + if (isRunningInCI()) { + options.optimizeForSmallDb(); + // Disable fallocate calls to avoid issues with disk space + options.setAllowFAllocate(false); + // Set WAL size limits to avoid excessive disk + options.setMaxTotalWalSize(2 * 1024 * 1024); + // Set recycle log file + options.setRecycleLogFileNum(1); + // Enable creation of missing column families + options.setCreateMissingColumnFamilies(true); + // Set max background flushes to 1 to reduce resource usage + options.setMaxBackgroundFlushes(1); + } + + return options; + } + + private static boolean isRunningInCI() { + return Arrays.stream(CI_ENVIRONMENT_VARIABLES).anyMatch(System.getenv()::containsKey); + } } diff --git a/common/src/main/java/org/tron/common/utils/ByteArray.java b/common/src/main/java/org/tron/common/utils/ByteArray.java index b77dd310380..d0ac4cbaddf 100644 --- a/common/src/main/java/org/tron/common/utils/ByteArray.java +++ b/common/src/main/java/org/tron/common/utils/ByteArray.java @@ -14,7 +14,7 @@ import org.apache.commons.lang3.ArrayUtils; import org.apache.commons.lang3.StringUtils; import org.bouncycastle.util.encoders.Hex; -import org.tron.core.exception.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; /* * Copyright (c) [2016] [ ] diff --git a/common/src/main/java/org/tron/common/utils/StringUtil.java b/common/src/main/java/org/tron/common/utils/StringUtil.java index ce3e95af46e..412a70d7f9c 100644 --- a/common/src/main/java/org/tron/common/utils/StringUtil.java +++ b/common/src/main/java/org/tron/common/utils/StringUtil.java @@ -44,4 +44,21 @@ public static String createReadableString(ByteString string) { public static ByteString hexString2ByteString(String hexString) { return ByteString.copyFrom(ByteArray.fromHexString(hexString)); } + + public static boolean isHexadecimal(String str) { + if (str == null || str.isEmpty()) { + return false; + } + if (str.length() % 2 != 0) { + return false; + } + + for (int i = 0; i < str.length(); i++) { + char c = str.charAt(i); + if (Character.digit(c, 16) == -1) { + return false; + } + } + return true; + } } diff --git a/common/src/main/java/org/tron/common/utils/TimeoutInterceptor.java b/common/src/main/java/org/tron/common/utils/TimeoutInterceptor.java new file mode 100644 index 00000000000..07b00e861e8 --- /dev/null +++ b/common/src/main/java/org/tron/common/utils/TimeoutInterceptor.java @@ -0,0 +1,30 @@ +package org.tron.common.utils; + +import io.grpc.CallOptions; +import io.grpc.Channel; +import io.grpc.ClientCall; +import io.grpc.ClientInterceptor; +import io.grpc.MethodDescriptor; +import java.util.concurrent.TimeUnit; + + +public class TimeoutInterceptor implements ClientInterceptor { + + private final long timeout; + + /** + * @param timeout ms + */ + public TimeoutInterceptor(long timeout) { + this.timeout = timeout; + } + + @Override + public ClientCall interceptCall( + MethodDescriptor method, + CallOptions callOptions, + Channel next) { + return next.newCall(method, + callOptions.withDeadlineAfter(timeout, TimeUnit.MILLISECONDS)); + } +} diff --git a/common/src/main/java/org/tron/core/Constant.java b/common/src/main/java/org/tron/core/Constant.java index c5a8a02fb4e..47331808a5b 100644 --- a/common/src/main/java/org/tron/core/Constant.java +++ b/common/src/main/java/org/tron/core/Constant.java @@ -24,6 +24,10 @@ public class Constant { public static final int NODE_TYPE_FULL_NODE = 0; public static final int NODE_TYPE_LIGHT_NODE = 1; + // DB NAME + public static final String MARKET_PAIR_PRICE_TO_ORDER = "market_pair_price_to_order"; + // DB NAME + // config for transaction public static final long TRANSACTION_MAX_BYTE_SIZE = 500 * 1_024L; public static final int CREATE_ACCOUNT_TRANSACTION_MIN_BYTE_SIZE = 500; @@ -39,6 +43,9 @@ public class Constant { public static final long MAX_CONTRACT_RESULT_SIZE = 2L; public static final long PB_DEFAULT_ENERGY_LIMIT = 0L; public static final long CREATOR_DEFAULT_ENERGY_LIMIT = 1000 * 10_000L; + public static final long MIN_PROPOSAL_EXPIRE_TIME = 0L; // 0 ms + public static final long MAX_PROPOSAL_EXPIRE_TIME = 31536003000L; // ms of 365 days + 3000 ms + public static final long DEFAULT_PROPOSAL_EXPIRE_TIME = 259200000L; // ms of 3 days // Numbers @@ -147,6 +154,7 @@ public class Constant { public static final String NODE_JSONRPC_HTTP_PBFT_PORT = "node.jsonrpc.httpPBFTPort"; public static final String NODE_JSONRPC_MAX_BLOCK_RANGE = "node.jsonrpc.maxBlockRange"; public static final String NODE_JSONRPC_MAX_SUB_TOPICS = "node.jsonrpc.maxSubTopics"; + public static final String NODE_JSONRPC_MAX_BLOCK_FILTER_NUM = "node.jsonrpc.maxBlockFilterNum"; public static final String NODE_DISABLED_API_LIST = "node.disabledApi"; @@ -156,6 +164,8 @@ public class Constant { public static final String NODE_RPC_MAX_CONCURRENT_CALLS_PER_CONNECTION = "node.rpc.maxConcurrentCallsPerConnection"; public static final String NODE_RPC_FLOW_CONTROL_WINDOW = "node.rpc.flowControlWindow"; public static final String NODE_RPC_MAX_CONNECTION_IDLE_IN_MILLIS = "node.rpc.maxConnectionIdleInMillis"; + public static final String NODE_RPC_MAX_RST_STREAM = "node.rpc.maxRstStream"; + public static final String NODE_RPC_SECONDS_PER_WINDOW = "node.rpc.secondsPerWindow"; public static final String NODE_PRODUCED_TIMEOUT = "node.blockProducedTimeOut"; public static final String NODE_MAX_HTTP_CONNECT_NUMBER = "node.maxHttpConnectNumber"; @@ -251,6 +261,9 @@ public class Constant { public static final String NODE_FULLNODE_ALLOW_SHIELDED_TRANSACTION = "node" + ".fullNodeAllowShieldedTransaction"; + public static final String ALLOW_SHIELDED_TRANSACTION_API = "node" + + ".allowShieldedTransactionApi"; + public static final String NODE_ZEN_TOKENID = "node.zenTokenId"; public static final String COMMITTEE_ALLOW_PROTO_FILTER_NUM = "committee.allowProtoFilterNum"; @@ -318,6 +331,9 @@ public class Constant { public static final String RATE_LIMITER_HTTP = "rate.limiter.http"; public static final String RATE_LIMITER_RPC = "rate.limiter.rpc"; + public static final String RATE_LIMITER_P2P_SYNC_BLOCK_CHAIN = "rate.limiter.p2p.syncBlockChain"; + public static final String RATE_LIMITER_P2P_FETCH_INV_DATA = "rate.limiter.p2p.fetchInvData"; + public static final String RATE_LIMITER_P2P_DISCONNECT = "rate.limiter.p2p.disconnect"; public static final String SEED_NODE_IP_LIST = "seed.node.ip.list"; public static final String NODE_METRICS_ENABLE = "node.metricsEnable"; @@ -405,4 +421,7 @@ public class Constant { public static final String COMMITTEE_ALLOW_TVM_CANCUN = "committee.allowTvmCancun"; public static final String COMMITTEE_ALLOW_TVM_BLOB = "committee.allowTvmBlob"; + + public static final String COMMITTEE_PROPOSAL_EXPIRE_TIME = "committee.proposalExpireTime"; + } diff --git a/common/src/main/java/org/tron/core/config/CommonConfig.java b/common/src/main/java/org/tron/core/config/CommonConfig.java index 4d17a0faedd..b258a4cf3f5 100644 --- a/common/src/main/java/org/tron/core/config/CommonConfig.java +++ b/common/src/main/java/org/tron/core/config/CommonConfig.java @@ -21,10 +21,8 @@ import org.springframework.context.annotation.ComponentScan; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.EnableAspectJAutoProxy; -import org.springframework.transaction.annotation.EnableTransactionManagement; @Configuration -@EnableTransactionManagement @EnableAspectJAutoProxy @ComponentScan(basePackages = "org.tron") public class CommonConfig { diff --git a/common/src/main/java/org/tron/core/config/Parameter.java b/common/src/main/java/org/tron/core/config/Parameter.java index a71dc58e8bd..8ec27ed3eb5 100644 --- a/common/src/main/java/org/tron/core/config/Parameter.java +++ b/common/src/main/java/org/tron/core/config/Parameter.java @@ -26,7 +26,9 @@ public enum ForkBlockVersionEnum { VERSION_4_7_4(29, 1596780000000L, 80), VERSION_4_7_5(30, 1596780000000L, 80), VERSION_4_7_7(31, 1596780000000L, 80), - VERSION_4_8_0(32, 1596780000000L, 80); + VERSION_4_8_0(32, 1596780000000L, 80), + VERSION_4_8_0_1(33, 1596780000000L, 70), + VERSION_4_8_1(34, 1596780000000L, 80); // if add a version, modify BLOCK_VERSION simultaneously @Getter @@ -75,7 +77,7 @@ public class ChainConstant { public static final int SINGLE_REPEAT = 1; public static final int BLOCK_FILLED_SLOTS_NUMBER = 128; public static final int MAX_FROZEN_NUMBER = 1; - public static final int BLOCK_VERSION = 32; + public static final int BLOCK_VERSION = 34; public static final long FROZEN_PERIOD = 86_400_000L; public static final long DELEGATE_PERIOD = 3 * 86_400_000L; public static final long TRX_PRECISION = 1000_000L; @@ -107,6 +109,7 @@ public class DatabaseConstants { public static final int PROPOSAL_COUNT_LIMIT_MAX = 1000; public static final int EXCHANGE_COUNT_LIMIT_MAX = 1000; public static final int MARKET_COUNT_LIMIT_MAX = 1000; + public static final int WITNESS_COUNT_LIMIT_MAX = 1000; } public class AdaptiveResourceLimitConstants { diff --git a/common/src/main/java/org/tron/core/config/args/Storage.java b/common/src/main/java/org/tron/core/config/args/Storage.java index 9cf6eb6bab1..655b6b779fe 100644 --- a/common/src/main/java/org/tron/core/config/args/Storage.java +++ b/common/src/main/java/org/tron/core/config/args/Storage.java @@ -25,9 +25,11 @@ import java.util.stream.Collectors; import lombok.Getter; import lombok.Setter; +import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang3.StringUtils; import org.iq80.leveldb.CompressionType; import org.iq80.leveldb.Options; +import org.tron.common.arch.Arch; import org.tron.common.cache.CacheStrategies; import org.tron.common.cache.CacheType; import org.tron.common.utils.DbOptionalsUtils; @@ -42,6 +44,7 @@ * @version 1.0 * @since 2018/5/25 */ +@Slf4j(topic = "db") public class Storage { /** @@ -87,6 +90,7 @@ public class Storage { * Default values of directory */ private static final String DEFAULT_DB_ENGINE = "LEVELDB"; + private static final String ROCKS_DB_ENGINE = "ROCKSDB"; private static final boolean DEFAULT_DB_SYNC = false; private static final boolean DEFAULT_EVENT_SUBSCRIBE_CONTRACT_PARSE = true; private static final String DEFAULT_DB_DIRECTORY = "database"; @@ -171,6 +175,11 @@ public class Storage { private final Map dbRoots = Maps.newConcurrentMap(); public static String getDbEngineFromConfig(final Config config) { + if (Arch.isArm64()) { + // if is arm64 but config is leveldb, should throw exception? + logger.warn("Arm64 architecture detected, using RocksDB as db engine, ignore config."); + return ROCKS_DB_ENGINE; + } return config.hasPath(DB_ENGINE_CONFIG_KEY) ? config.getString(DB_ENGINE_CONFIG_KEY) : DEFAULT_DB_ENGINE; } diff --git a/common/src/main/java/org/tron/core/exception/MaintenanceUnavailableException.java b/common/src/main/java/org/tron/core/exception/MaintenanceUnavailableException.java new file mode 100644 index 00000000000..1911f84a616 --- /dev/null +++ b/common/src/main/java/org/tron/core/exception/MaintenanceUnavailableException.java @@ -0,0 +1,20 @@ +package org.tron.core.exception; + +/** + * Maintenance unavailable exception - thrown when service is in maintenance state + * Please try again later + */ +public class MaintenanceUnavailableException extends TronException { + + public MaintenanceUnavailableException() { + super(); + } + + public MaintenanceUnavailableException(String message) { + super(message); + } + + public MaintenanceUnavailableException(String message, Throwable cause) { + super(message, cause); + } +} \ No newline at end of file diff --git a/common/src/main/java/org/tron/core/exception/P2pException.java b/common/src/main/java/org/tron/core/exception/P2pException.java index 00d82e9fbf7..eae830627c2 100644 --- a/common/src/main/java/org/tron/core/exception/P2pException.java +++ b/common/src/main/java/org/tron/core/exception/P2pException.java @@ -52,6 +52,7 @@ public enum TypeEnum { PROTOBUF_ERROR(14, "protobuf inconsistent"), BLOCK_SIGN_ERROR(15, "block sign error"), BLOCK_MERKLE_ERROR(16, "block merkle error"), + RATE_LIMIT_EXCEEDED(17, "rate limit exceeded"), DEFAULT(100, "default exception"); diff --git a/common/src/main/java/org/tron/core/exception/TronError.java b/common/src/main/java/org/tron/core/exception/TronError.java index 9d11d249476..f407c6dfe3c 100644 --- a/common/src/main/java/org/tron/core/exception/TronError.java +++ b/common/src/main/java/org/tron/core/exception/TronError.java @@ -47,7 +47,9 @@ public enum ErrCode { LOG_LOAD(1), WITNESS_INIT(1), RATE_LIMITER_INIT(1), - SOLID_NODE_INIT(0); + SOLID_NODE_INIT(0), + PARAMETER_INIT(1), + JDK_VERSION(1); private final int code; diff --git a/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcExceedLimitException.java b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcExceedLimitException.java new file mode 100644 index 00000000000..9e6f59d4636 --- /dev/null +++ b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcExceedLimitException.java @@ -0,0 +1,16 @@ +package org.tron.core.exception.jsonrpc; + +public class JsonRpcExceedLimitException extends JsonRpcException { + + public JsonRpcExceedLimitException() { + super(); + } + + public JsonRpcExceedLimitException(String message) { + super(message); + } + + public JsonRpcExceedLimitException(String message, Throwable cause) { + super(message, cause); + } +} \ No newline at end of file diff --git a/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcException.java b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcException.java new file mode 100644 index 00000000000..7ed42d101d5 --- /dev/null +++ b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcException.java @@ -0,0 +1,32 @@ +package org.tron.core.exception.jsonrpc; + +import lombok.Getter; +import org.tron.core.exception.TronException; + +@Getter +public class JsonRpcException extends TronException { + private Object data = null; + + public JsonRpcException() { + super(); + report(); + } + + public JsonRpcException(String message, Object data) { + super(message); + this.data = data; + report(); + } + + public JsonRpcException(String message) { + super(message); + report(); + } + + public JsonRpcException(String message, Throwable cause) { + super(message, cause); + report(); + } + + +} diff --git a/common/src/main/java/org/tron/core/exception/JsonRpcInternalException.java b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcInternalException.java similarity index 53% rename from common/src/main/java/org/tron/core/exception/JsonRpcInternalException.java rename to common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcInternalException.java index 12310e67747..904449866ae 100644 --- a/common/src/main/java/org/tron/core/exception/JsonRpcInternalException.java +++ b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcInternalException.java @@ -1,6 +1,6 @@ -package org.tron.core.exception; +package org.tron.core.exception.jsonrpc; -public class JsonRpcInternalException extends TronException { +public class JsonRpcInternalException extends JsonRpcException { public JsonRpcInternalException() { super(); @@ -13,4 +13,8 @@ public JsonRpcInternalException(String message) { public JsonRpcInternalException(String message, Throwable cause) { super(message, cause); } + + public JsonRpcInternalException(String message, Object data) { + super(message, data); + } } \ No newline at end of file diff --git a/common/src/main/java/org/tron/core/exception/JsonRpcInvalidParamsException.java b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcInvalidParamsException.java similarity index 68% rename from common/src/main/java/org/tron/core/exception/JsonRpcInvalidParamsException.java rename to common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcInvalidParamsException.java index adf205a309a..55ee15521e1 100644 --- a/common/src/main/java/org/tron/core/exception/JsonRpcInvalidParamsException.java +++ b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcInvalidParamsException.java @@ -1,6 +1,6 @@ -package org.tron.core.exception; +package org.tron.core.exception.jsonrpc; -public class JsonRpcInvalidParamsException extends TronException { +public class JsonRpcInvalidParamsException extends JsonRpcException { public JsonRpcInvalidParamsException() { super(); diff --git a/common/src/main/java/org/tron/core/exception/JsonRpcInvalidRequestException.java b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcInvalidRequestException.java similarity index 69% rename from common/src/main/java/org/tron/core/exception/JsonRpcInvalidRequestException.java rename to common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcInvalidRequestException.java index 2689bff0cd2..32bd11a3ed9 100644 --- a/common/src/main/java/org/tron/core/exception/JsonRpcInvalidRequestException.java +++ b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcInvalidRequestException.java @@ -1,6 +1,6 @@ -package org.tron.core.exception; +package org.tron.core.exception.jsonrpc; -public class JsonRpcInvalidRequestException extends TronException { +public class JsonRpcInvalidRequestException extends JsonRpcException { public JsonRpcInvalidRequestException() { super(); diff --git a/common/src/main/java/org/tron/core/exception/JsonRpcMethodNotFoundException.java b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcMethodNotFoundException.java similarity index 68% rename from common/src/main/java/org/tron/core/exception/JsonRpcMethodNotFoundException.java rename to common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcMethodNotFoundException.java index d8e18168d9d..406a51f45bd 100644 --- a/common/src/main/java/org/tron/core/exception/JsonRpcMethodNotFoundException.java +++ b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcMethodNotFoundException.java @@ -1,6 +1,6 @@ -package org.tron.core.exception; +package org.tron.core.exception.jsonrpc; -public class JsonRpcMethodNotFoundException extends TronException { +public class JsonRpcMethodNotFoundException extends JsonRpcException { public JsonRpcMethodNotFoundException() { super(); diff --git a/common/src/main/java/org/tron/core/exception/JsonRpcTooManyResultException.java b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcTooManyResultException.java similarity index 69% rename from common/src/main/java/org/tron/core/exception/JsonRpcTooManyResultException.java rename to common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcTooManyResultException.java index e8e183d49c1..03bc089b1c7 100644 --- a/common/src/main/java/org/tron/core/exception/JsonRpcTooManyResultException.java +++ b/common/src/main/java/org/tron/core/exception/jsonrpc/JsonRpcTooManyResultException.java @@ -1,6 +1,6 @@ -package org.tron.core.exception; +package org.tron.core.exception.jsonrpc; -public class JsonRpcTooManyResultException extends TronException { +public class JsonRpcTooManyResultException extends JsonRpcException { public JsonRpcTooManyResultException() { super(); diff --git a/common/src/main/java/org/tron/core/vm/config/VMConfig.java b/common/src/main/java/org/tron/core/vm/config/VMConfig.java index 2bdbeb78b92..578827b2f8c 100644 --- a/common/src/main/java/org/tron/core/vm/config/VMConfig.java +++ b/common/src/main/java/org/tron/core/vm/config/VMConfig.java @@ -59,6 +59,8 @@ public class VMConfig { private static boolean ALLOW_TVM_BLOB = false; + private static boolean ALLOW_TVM_SELFDESTRUCT_RESTRICTION = false; + private VMConfig() { } @@ -166,6 +168,10 @@ public static void initAllowTvmBlob(long allow) { ALLOW_TVM_BLOB = allow == 1; } + public static void initAllowTvmSelfdestructRestriction(long allow) { + ALLOW_TVM_SELFDESTRUCT_RESTRICTION = allow == 1; + } + public static boolean getEnergyLimitHardFork() { return CommonParameter.ENERGY_LIMIT_HARD_FORK; } @@ -261,4 +267,8 @@ public static boolean disableJavaLangMath() { public static boolean allowTvmBlob() { return ALLOW_TVM_BLOB; } + + public static boolean allowTvmSelfdestructRestriction() { + return ALLOW_TVM_SELFDESTRUCT_RESTRICTION; + } } diff --git a/crypto/src/main/java/org/tron/common/crypto/ECKey.java b/crypto/src/main/java/org/tron/common/crypto/ECKey.java index 5fe8b9ef359..d0a6048aca1 100644 --- a/crypto/src/main/java/org/tron/common/crypto/ECKey.java +++ b/crypto/src/main/java/org/tron/common/crypto/ECKey.java @@ -55,6 +55,7 @@ import org.tron.common.crypto.jce.ECKeyPairGenerator; import org.tron.common.crypto.jce.TronCastleProvider; import org.tron.common.utils.BIUtil; +import org.tron.common.utils.ByteArray; import org.tron.common.utils.ByteUtil; @Slf4j(topic = "crypto") @@ -285,7 +286,7 @@ public static ECKey fromPrivate(BigInteger privKey) { * @return - */ public static ECKey fromPrivate(byte[] privKeyBytes) { - if (Objects.isNull(privKeyBytes)) { + if (ByteArray.isEmpty(privKeyBytes)) { return null; } return fromPrivate(new BigInteger(1, privKeyBytes)); diff --git a/crypto/src/main/java/org/tron/common/crypto/Rsv.java b/crypto/src/main/java/org/tron/common/crypto/Rsv.java new file mode 100644 index 00000000000..15c8498e836 --- /dev/null +++ b/crypto/src/main/java/org/tron/common/crypto/Rsv.java @@ -0,0 +1,26 @@ +package org.tron.common.crypto; + + +import java.util.Arrays; +import lombok.AllArgsConstructor; +import lombok.Getter; + +@Getter +@AllArgsConstructor +public class Rsv { + + private final byte[] r; + private final byte[] s; + private final byte v; + + + public static Rsv fromSignature(byte[] sign) { + byte[] r = Arrays.copyOfRange(sign, 0, 32); + byte[] s = Arrays.copyOfRange(sign, 32, 64); + byte v = sign[64]; + if (v < 27) { + v += 27; //revId -> v + } + return new Rsv(r, s, v); + } +} diff --git a/crypto/src/main/java/org/tron/common/crypto/sm2/SM2.java b/crypto/src/main/java/org/tron/common/crypto/sm2/SM2.java index c6aebba385a..b1d349efad3 100644 --- a/crypto/src/main/java/org/tron/common/crypto/sm2/SM2.java +++ b/crypto/src/main/java/org/tron/common/crypto/sm2/SM2.java @@ -41,6 +41,7 @@ import org.tron.common.crypto.SignatureInterface; import org.tron.common.crypto.jce.ECKeyFactory; import org.tron.common.crypto.jce.TronCastleProvider; +import org.tron.common.utils.ByteArray; import org.tron.common.utils.ByteUtil; /** @@ -247,7 +248,7 @@ public static SM2 fromPrivate(BigInteger privKey) { * @return - */ public static SM2 fromPrivate(byte[] privKeyBytes) { - if (Objects.isNull(privKeyBytes)) { + if (ByteArray.isEmpty(privKeyBytes)) { return null; } return fromPrivate(new BigInteger(1, privKeyBytes)); diff --git a/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp12.java b/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp12.java index 26ea708fbe4..f144656b0ba 100644 --- a/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp12.java +++ b/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp12.java @@ -356,7 +356,7 @@ public boolean equals(Object o) { @Override public int hashCode() { - return new Integer(a.hashCode() + b.hashCode()).hashCode(); + return a.hashCode() + b.hashCode(); } @Override diff --git a/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp2.java b/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp2.java index ba2a1ceb477..162a5b13b30 100644 --- a/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp2.java +++ b/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp2.java @@ -164,7 +164,7 @@ public boolean equals(Object o) { @Override public int hashCode() { - return new Integer(a.hashCode() + b.hashCode()).hashCode(); + return a.hashCode() + b.hashCode(); } Fp2 frobeniusMap(int power) { diff --git a/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp6.java b/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp6.java index 0636cc334f1..fb863bb6b34 100644 --- a/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp6.java +++ b/crypto/src/main/java/org/tron/common/crypto/zksnark/Fp6.java @@ -246,6 +246,6 @@ public boolean equals(Object o) { @Override public int hashCode() { - return new Integer(a.hashCode() + b.hashCode() + c.hashCode()).hashCode(); + return a.hashCode() + b.hashCode() + c.hashCode(); } } diff --git a/docker/arm64/Dockerfile b/docker/arm64/Dockerfile new file mode 100644 index 00000000000..6435faf7ead --- /dev/null +++ b/docker/arm64/Dockerfile @@ -0,0 +1,33 @@ +FROM arm64v8/eclipse-temurin:17 + +ENV TMP_DIR="/tron-build" +ENV BASE_DIR="/java-tron" + +RUN set -o errexit -o nounset \ + && apt-get update \ + && apt-get -y install git p7zip-full wget libtcmalloc-minimal4 \ + && echo "git clone" \ + && mkdir -p $TMP_DIR \ + && cd $TMP_DIR \ + && git clone https://github.com/tronprotocol/java-tron.git \ + && cd java-tron \ + && git checkout master \ + && ./gradlew clean build -x test -x check --no-daemon \ + && cd build/distributions \ + && 7za x -y java-tron-1.0.0.zip \ + && mv java-tron-1.0.0 $BASE_DIR \ + && rm -rf $TMP_DIR \ + && rm -rf ~/.gradle \ + && apt-get clean \ + && rm -rf /var/lib/apt/lists/* + +ENV LD_PRELOAD="/usr/lib/aarch64-linux-gnu/libtcmalloc_minimal.so.4" +ENV TCMALLOC_RELEASE_RATE=10 + +RUN wget -P $BASE_DIR/config https://raw.githubusercontent.com/tronprotocol/tron-deployment/master/main_net_config.conf + +COPY docker-entrypoint.sh $BASE_DIR/bin + +WORKDIR $BASE_DIR + +ENTRYPOINT ["./bin/docker-entrypoint.sh"] \ No newline at end of file diff --git a/framework/build.gradle b/framework/build.gradle index 0f04685f2d8..59d070e066d 100644 --- a/framework/build.gradle +++ b/framework/build.gradle @@ -38,15 +38,11 @@ dependencies { //local libraries implementation fileTree(dir: 'libs', include: '*.jar') // end local libraries - testImplementation group: 'org.hamcrest', name: 'hamcrest-junit', version: '1.0.0.1' - - implementation group: 'com.google.inject', name: 'guice', version: '4.1.0' implementation group: 'io.dropwizard.metrics', name: 'metrics-core', version: '3.1.2' implementation group: 'com.github.davidb', name: 'metrics-influxdb', version: '0.8.2' - implementation group: 'com.carrotsearch', name: 'java-sizeof', version: '0.0.5' // http - implementation 'org.eclipse.jetty:jetty-server:9.4.53.v20231009' - implementation 'org.eclipse.jetty:jetty-servlet:9.4.53.v20231009' + implementation 'org.eclipse.jetty:jetty-server:9.4.57.v20241219' + implementation 'org.eclipse.jetty:jetty-servlet:9.4.57.v20241219' implementation 'com.alibaba:fastjson:1.2.83' // end http @@ -56,14 +52,11 @@ dependencies { // https://mvnrepository.com/artifact/javax.portlet/portlet-api compileOnly group: 'javax.portlet', name: 'portlet-api', version: '3.0.1' - implementation "io.vavr:vavr:0.9.2" implementation (group: 'org.pf4j', name: 'pf4j', version: '3.10.0') { exclude group: "org.slf4j", module: "slf4j-api" } - testImplementation group: 'org.springframework', name: 'spring-test', version: '5.2.0.RELEASE' - testImplementation group: 'org.springframework', name: 'spring-web', version: '5.2.0.RELEASE' - + testImplementation group: 'org.springframework', name: 'spring-test', version: "${springVersion}" implementation group: 'org.zeromq', name: 'jeromq', version: '0.5.3' api project(":chainbase") api project(":protocol") @@ -123,6 +116,17 @@ test { destinationFile = file("$buildDir/jacoco/jacocoTest.exec") classDumpDir = file("$buildDir/jacoco/classpathdumps") } + if (rootProject.archInfo.isArm64) { + exclude { element -> + element.file.name.toLowerCase().contains('leveldb') + } + filter { + excludeTestsMatching '*.*leveldb*' + excludeTestsMatching '*.*Leveldb*' + excludeTestsMatching '*.*LevelDB*' + excludeTestsMatching '*.*LevelDb*' + } + } if (isWindows()) { exclude '**/ShieldedTransferActuatorTest.class' exclude '**/BackupDbUtilTest.class' @@ -134,6 +138,7 @@ test { } maxHeapSize = "1024m" doFirst { + // Restart the JVM after every 100 tests to avoid memory leaks and ensure test isolation forkEvery = 100 jvmArgs "-XX:MetaspaceSize=128m","-XX:MaxMetaspaceSize=256m", "-XX:+UseG1GC" } @@ -158,7 +163,7 @@ def binaryRelease(taskName, jarName, mainClass) { // explicit_dependency dependsOn (project(':actuator').jar, project(':consensus').jar, project(':chainbase').jar, - project(':crypto').jar, project(':common').jar, project(':protocol').jar) + project(':crypto').jar, project(':common').jar, project(':protocol').jar, project(':platform').jar) from { configurations.runtimeClasspath.collect { @@ -204,8 +209,7 @@ def createScript(project, mainClass, name) { } } } - -applicationDistribution.from("../gradle/java-tron.vmoptions") { +applicationDistribution.from(rootProject.archInfo.VMOptions) { into "bin" } //distZip { @@ -219,35 +223,12 @@ startScripts.enabled = false run.enabled = false tasks.distTar.enabled = false -createScript(project, 'org.tron.program.SolidityNode', 'SolidityNode') createScript(project, 'org.tron.program.FullNode', 'FullNode') -createScript(project, 'org.tron.program.KeystoreFactory', 'KeystoreFactory') -createScript(project, 'org.tron.program.DBConvert', 'DBConvert') - def releaseBinary = hasProperty('binaryRelease') ? getProperty('binaryRelease') : 'true' -def skipSolidity = hasProperty('skipSolidity') ? true : false -def skipKeystore = hasProperty('skipKeystore') ? true : false -def skipConvert = hasProperty('skipConvert') ? true : false -def skipAll = hasProperty('skipAll') ? true : false if (releaseBinary == 'true') { artifacts { archives(binaryRelease('buildFullNodeJar', 'FullNode', 'org.tron.program.FullNode')) } - if (!skipAll) { - if (!skipSolidity) { - artifacts { - archives(binaryRelease('buildSolidityNodeJar', 'SolidityNode', 'org.tron.program.SolidityNode'))} - } - if (!skipKeystore) { - artifacts { - archives(binaryRelease('buildKeystoreFactoryJar', 'KeystoreFactory', 'org.tron.program.KeystoreFactory'))} - } - if (!skipConvert) { - artifacts { - archives(binaryRelease('buildDBConvertJar', 'DBConvert', 'org.tron.program.DBConvert'))} - } - } - } task copyToParent(type: Copy) { diff --git a/framework/src/main/java/org/tron/common/application/RpcService.java b/framework/src/main/java/org/tron/common/application/RpcService.java index 2d118806e2c..c398b71ae41 100644 --- a/framework/src/main/java/org/tron/common/application/RpcService.java +++ b/framework/src/main/java/org/tron/common/application/RpcService.java @@ -19,6 +19,7 @@ import io.grpc.netty.NettyServerBuilder; import io.grpc.protobuf.services.ProtoReflectionService; import java.util.concurrent.CompletableFuture; +import java.util.concurrent.ExecutorService; import java.util.concurrent.TimeUnit; import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Autowired; @@ -34,6 +35,7 @@ public abstract class RpcService extends AbstractService { private Server apiServer; + private ExecutorService executorService; protected String executorName; @Autowired @@ -58,7 +60,24 @@ public void innerStart() throws Exception { @Override public void innerStop() throws Exception { if (this.apiServer != null) { - this.apiServer.shutdown().awaitTermination(5, TimeUnit.SECONDS); + this.apiServer.shutdown(); + try { + if (!this.apiServer.awaitTermination(5, TimeUnit.SECONDS)) { + logger.warn("gRPC server did not shutdown gracefully, forcing shutdown"); + this.apiServer.shutdownNow(); + } + } catch (InterruptedException e) { + logger.warn("Interrupted while waiting for gRPC server shutdown", e); + this.apiServer.shutdownNow(); + Thread.currentThread().interrupt(); + } + } + + // Close executor + if (this.executorService != null) { + ExecutorServiceManager.shutdownAndAwaitTermination( + this.executorService, this.executorName); + this.executorService = null; } } @@ -76,9 +95,9 @@ protected NettyServerBuilder initServerBuilder() { NettyServerBuilder serverBuilder = NettyServerBuilder.forPort(this.port); CommonParameter parameter = Args.getInstance(); if (parameter.getRpcThreadNum() > 0) { - serverBuilder = serverBuilder - .executor(ExecutorServiceManager.newFixedThreadPool( - this.executorName, parameter.getRpcThreadNum())); + this.executorService = ExecutorServiceManager.newFixedThreadPool( + this.executorName, parameter.getRpcThreadNum()); + serverBuilder = serverBuilder.executor(this.executorService); } // Set configs from config.conf or default value serverBuilder @@ -88,6 +107,10 @@ protected NettyServerBuilder initServerBuilder() { .maxConnectionAge(parameter.getMaxConnectionAgeInMillis(), TimeUnit.MILLISECONDS) .maxInboundMessageSize(parameter.getMaxMessageSize()) .maxHeaderListSize(parameter.getMaxHeaderListSize()); + if (parameter.getRpcMaxRstStream() > 0 && parameter.getRpcSecondsPerWindow() > 0) { + serverBuilder.maxRstFramesPerWindow( + parameter.getRpcMaxRstStream(), parameter.getRpcSecondsPerWindow()); + } if (parameter.isRpcReflectionServiceEnable()) { serverBuilder.addService(ProtoReflectionService.newInstance()); diff --git a/framework/src/main/java/org/tron/common/application/TronApplicationContext.java b/framework/src/main/java/org/tron/common/application/TronApplicationContext.java index 64edec77c9c..6c934528f4f 100644 --- a/framework/src/main/java/org/tron/common/application/TronApplicationContext.java +++ b/framework/src/main/java/org/tron/common/application/TronApplicationContext.java @@ -13,8 +13,10 @@ public TronApplicationContext(DefaultListableBeanFactory beanFactory) { super(beanFactory); } + //only used for testcase public TronApplicationContext(Class... annotatedClasses) { super(annotatedClasses); + this.registerShutdownHook(); } public TronApplicationContext(String... basePackages) { diff --git a/framework/src/main/java/org/tron/common/backup/BackupManager.java b/framework/src/main/java/org/tron/common/backup/BackupManager.java index 0c4a3e60dfd..a8812a62bb4 100644 --- a/framework/src/main/java/org/tron/common/backup/BackupManager.java +++ b/framework/src/main/java/org/tron/common/backup/BackupManager.java @@ -126,7 +126,7 @@ public void handleEvent(UdpEvent udpEvent) { logger.warn("Receive keep alive message from {} is not my member", sender.getHostString()); return; } - + logger.info("Receive keep alive message from {}", sender); lastKeepAliveTime = System.currentTimeMillis(); KeepAliveMessage keepAliveMessage = (KeepAliveMessage) msg; diff --git a/framework/src/main/java/org/tron/common/logsfilter/EventPluginLoader.java b/framework/src/main/java/org/tron/common/logsfilter/EventPluginLoader.java index 0a7a5ac3a76..7061b2e9d57 100644 --- a/framework/src/main/java/org/tron/common/logsfilter/EventPluginLoader.java +++ b/framework/src/main/java/org/tron/common/logsfilter/EventPluginLoader.java @@ -9,7 +9,6 @@ import java.util.Objects; import java.util.Set; import java.util.stream.Collectors; - import lombok.Getter; import lombok.extern.slf4j.Slf4j; import org.bouncycastle.util.encoders.Hex; @@ -26,7 +25,6 @@ import org.tron.common.logsfilter.trigger.SolidityTrigger; import org.tron.common.logsfilter.trigger.TransactionLogTrigger; import org.tron.common.logsfilter.trigger.Trigger; -import org.tron.common.utils.JsonUtil; @Slf4j public class EventPluginLoader { @@ -140,7 +138,7 @@ public static boolean matchFilter(ContractTrigger trigger) { private static boolean filterContractAddress(ContractTrigger trigger, List addressList) { addressList = addressList.stream().filter(item -> - org.apache.commons.lang3.StringUtils.isNotEmpty(item)) + org.apache.commons.lang3.StringUtils.isNotEmpty(item)) .collect(Collectors.toList()); if (Objects.isNull(addressList) || addressList.isEmpty()) { return true; @@ -173,7 +171,7 @@ private static boolean filterContractTopicList(ContractTrigger trigger, List(((ContractEventTrigger) trigger).getTopicMap().values()); } else if (trigger != null) { hset = trigger.getLogInfo().getClonedTopics() - .stream().map(Hex::toHexString).collect(Collectors.toSet()); + .stream().map(Hex::toHexString).collect(Collectors.toSet()); } for (String top : topList) { @@ -547,6 +545,10 @@ public boolean isBusy() { return false; } int queueSize = 0; + if (eventListeners == null || eventListeners.isEmpty()) { + // only occurs in mock test. TODO fix test + return false; + } for (IPluginEventListener listener : eventListeners) { try { queueSize += listener.getPendingSize(); diff --git a/framework/src/main/java/org/tron/core/Wallet.java b/framework/src/main/java/org/tron/core/Wallet.java index 8dfb18331ff..8c86f2f66ac 100755 --- a/framework/src/main/java/org/tron/core/Wallet.java +++ b/framework/src/main/java/org/tron/core/Wallet.java @@ -30,6 +30,7 @@ import static org.tron.core.config.Parameter.DatabaseConstants.EXCHANGE_COUNT_LIMIT_MAX; import static org.tron.core.config.Parameter.DatabaseConstants.MARKET_COUNT_LIMIT_MAX; import static org.tron.core.config.Parameter.DatabaseConstants.PROPOSAL_COUNT_LIMIT_MAX; +import static org.tron.core.config.Parameter.DatabaseConstants.WITNESS_COUNT_LIMIT_MAX; import static org.tron.core.services.jsonrpc.JsonRpcApiUtil.parseEnergyFee; import static org.tron.core.services.jsonrpc.TronJsonRpcImpl.EARLIEST_STR; import static org.tron.core.services.jsonrpc.TronJsonRpcImpl.FINALIZED_STR; @@ -44,6 +45,7 @@ import com.google.common.collect.DiscreteDomain; import com.google.common.collect.ImmutableList; import com.google.common.collect.Lists; +import com.google.common.collect.Maps; import com.google.common.collect.Range; import com.google.protobuf.ByteString; import com.google.protobuf.InvalidProtocolBufferException; @@ -59,6 +61,7 @@ import java.util.Map.Entry; import java.util.Objects; import java.util.Optional; +import java.util.stream.Collectors; import lombok.Getter; import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang3.ArrayUtils; @@ -161,6 +164,7 @@ import org.tron.core.capsule.TransactionInfoCapsule; import org.tron.core.capsule.TransactionResultCapsule; import org.tron.core.capsule.TransactionRetCapsule; +import org.tron.core.capsule.VotesCapsule; import org.tron.core.capsule.WitnessCapsule; import org.tron.core.capsule.utils.MarketUtils; import org.tron.core.config.args.Args; @@ -170,6 +174,7 @@ import org.tron.core.db.Manager; import org.tron.core.db.TransactionContext; import org.tron.core.db2.core.Chainbase; +import org.tron.core.db2.core.Chainbase.Cursor; import org.tron.core.exception.AccountResourceInsufficientException; import org.tron.core.exception.BadItemException; import org.tron.core.exception.ContractExeException; @@ -177,7 +182,7 @@ import org.tron.core.exception.DupTransactionException; import org.tron.core.exception.HeaderNotFound; import org.tron.core.exception.ItemNotFoundException; -import org.tron.core.exception.JsonRpcInvalidParamsException; +import org.tron.core.exception.MaintenanceUnavailableException; import org.tron.core.exception.NonUniqueObjectException; import org.tron.core.exception.PermissionException; import org.tron.core.exception.SignatureFormatException; @@ -188,6 +193,7 @@ import org.tron.core.exception.VMIllegalException; import org.tron.core.exception.ValidateSignatureException; import org.tron.core.exception.ZksnarkException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; import org.tron.core.net.TronNetDelegate; import org.tron.core.net.TronNetService; import org.tron.core.net.message.adv.TransactionMessage; @@ -201,6 +207,8 @@ import org.tron.core.store.MarketPairPriceToOrderStore; import org.tron.core.store.MarketPairToPriceStore; import org.tron.core.store.StoreFactory; +import org.tron.core.store.VotesStore; +import org.tron.core.store.WitnessStore; import org.tron.core.utils.TransactionUtil; import org.tron.core.vm.program.Program; import org.tron.core.zen.ShieldedTRC20ParametersBuilder; @@ -764,6 +772,110 @@ public WitnessList getWitnessList() { return builder.build(); } + public WitnessList getPaginatedNowWitnessList(long offset, long limit) throws + MaintenanceUnavailableException { + if (limit <= 0 || offset < 0) { + return null; + } + if (limit > WITNESS_COUNT_LIMIT_MAX) { + limit = WITNESS_COUNT_LIMIT_MAX; + } + + /* + In the maintenance period, the VoteStores will be cleared. + To avoid the race condition of VoteStores deleted but Witness vote counts not updated, + return retry error. + Only apply to requests that rely on the latest block, + which means the normal fullnode requests with HEAD cursor. + */ + boolean isMaintenance = chainBaseManager.getDynamicPropertiesStore().getStateFlag() == 1; + if (isMaintenance && !Args.getInstance().isSolidityNode() && getCursor() == Cursor.HEAD) { + String message = + "Service temporarily unavailable during maintenance period. Please try again later."; + throw new MaintenanceUnavailableException(message); + } + // It contains the final vote count at the end of the last epoch. + List witnessCapsuleList = chainBaseManager.getWitnessStore().getAllWitnesses(); + if (offset >= witnessCapsuleList.size()) { + return null; + } + + VotesStore votesStore = chainBaseManager.getVotesStore(); + // Count the vote changes for each witness in the current epoch, it is maybe negative. + Map countWitness = countVote(votesStore); + + // Iterate through the witness list to apply vote changes and calculate the real-time vote count + witnessCapsuleList.forEach(witnessCapsule -> { + long voteCount = countWitness.getOrDefault(witnessCapsule.getAddress(), 0L); + witnessCapsule.setVoteCount(witnessCapsule.getVoteCount() + voteCount); + }); + + // Use the same sorting logic as in the Maintenance period + WitnessStore.sortWitnesses(witnessCapsuleList, + chainBaseManager.getDynamicPropertiesStore().allowWitnessSortOptimization()); + + List sortedWitnessList = witnessCapsuleList.stream() + .skip(offset) + .limit(limit) + .collect(Collectors.toList()); + + WitnessList.Builder builder = WitnessList.newBuilder(); + sortedWitnessList.forEach(witnessCapsule -> + builder.addWitnesses(witnessCapsule.getInstance())); + + return builder.build(); + } + + /** + * Counts vote changes for witnesses in the current epoch. + * + * Vote count changes are tracked as follows: + * - Negative values for votes removed from previous witness in the last epoch + * - Positive values for votes added to new witness in the current epoch + * + * Example: + * an Account X had 100 votes for witness W1 in the previous epoch. + * In the current epoch, X changes votes to: + * - W2: 60 votes + * - W3: 80 votes + * + * Resulting vote changes: + * - W1: -100 (votes removed) + * - W2: +60 (new votes) + * - W3: +80 (new votes) + */ + private Map countVote(VotesStore votesStore) { + // Initialize a result map to store vote changes for each witness + Map countWitness = Maps.newHashMap(); + + // VotesStore is a key-value store, where the key is the address of the voter + Iterator> dbIterator = votesStore.iterator(); + + while (dbIterator.hasNext()) { + Entry next = dbIterator.next(); + VotesCapsule votes = next.getValue(); + + /** + * VotesCapsule contains two lists: + * - Old votes: Last votes from the previous epoch, updated in maintenance period + * - New votes: Latest votes in current epoch, updated after each vote transaction + */ + votes.getOldVotes().forEach(vote -> { + ByteString voteAddress = vote.getVoteAddress(); + long voteCount = vote.getVoteCount(); + countWitness.put(voteAddress, + countWitness.getOrDefault(voteAddress, 0L) - voteCount); + }); + votes.getNewVotes().forEach(vote -> { + ByteString voteAddress = vote.getVoteAddress(); + long voteCount = vote.getVoteCount(); + countWitness.put(voteAddress, + countWitness.getOrDefault(voteAddress, 0L) + voteCount); + }); + } + return countWitness; + } + public ProposalList getProposalList() { ProposalList.Builder builder = ProposalList.newBuilder(); List proposalCapsuleList = @@ -1387,6 +1499,16 @@ public Protocol.ChainParameters getChainParameters() { .setValue(dbManager.getDynamicPropertiesStore().getAllowTvmBlob()) .build()); + builder.addChainParameter(Protocol.ChainParameters.ChainParameter.newBuilder() + .setKey("getAllowTvmSelfdestructRestriction") + .setValue(dbManager.getDynamicPropertiesStore().getAllowTvmSelfdestructRestriction()) + .build()); + + builder.addChainParameter(Protocol.ChainParameters.ChainParameter.newBuilder() + .setKey("getProposalExpireTime") + .setValue(dbManager.getDynamicPropertiesStore().getProposalExpireTime()) + .build()); + return builder.build(); } @@ -1806,12 +1928,8 @@ public Exchange getExchangeById(ByteString exchangeId) { return null; } - private boolean getFullNodeAllowShieldedTransaction() { - return Args.getInstance().isFullNodeAllowShieldedTransactionArgs(); - } - - private void checkFullNodeAllowShieldedTransaction() throws ZksnarkException { - if (!getFullNodeAllowShieldedTransaction()) { + private void checkAllowShieldedTransactionApi() throws ZksnarkException { + if (!Args.getInstance().isAllowShieldedTransactionApi()) { throw new ZksnarkException(SHIELDED_ID_NOT_ALLOWED); } } @@ -1830,10 +1948,7 @@ public BytesMessage getNullifier(ByteString id) { } private long getBlockNumber(OutputPoint outPoint) - throws BadItemException, ZksnarkException { - if (!getFullNodeAllowShieldedTransaction()) { - throw new ZksnarkException(SHIELDED_ID_NOT_ALLOWED); - } + throws BadItemException { ByteString txId = outPoint.getHash(); long blockNum = chainBaseManager.getTransactionStore().getBlockNumber(txId.toByteArray()); @@ -1848,9 +1963,6 @@ private long getBlockNumber(OutputPoint outPoint) private IncrementalMerkleVoucherContainer createWitness(OutputPoint outPoint, Long blockNumber) throws ItemNotFoundException, BadItemException, InvalidProtocolBufferException, ZksnarkException { - if (!getFullNodeAllowShieldedTransaction()) { - throw new ZksnarkException(SHIELDED_ID_NOT_ALLOWED); - } ByteString txId = outPoint.getHash(); //Get the tree in blockNum-1 position @@ -1946,9 +2058,6 @@ private IncrementalMerkleVoucherContainer createWitness(OutputPoint outPoint, Lo private void updateWitnesses(List witnessList, long large, int synBlockNum) throws ItemNotFoundException, BadItemException, InvalidProtocolBufferException, ZksnarkException { - if (!getFullNodeAllowShieldedTransaction()) { - throw new ZksnarkException(SHIELDED_ID_NOT_ALLOWED); - } long start = large; long end = large + synBlockNum - 1; @@ -2022,10 +2131,7 @@ private void updateLowWitness(IncrementalMerkleVoucherContainer witness, long bl } } - private void validateInput(OutputPointInfo request) throws BadItemException, ZksnarkException { - if (!getFullNodeAllowShieldedTransaction()) { - throw new ZksnarkException(SHIELDED_ID_NOT_ALLOWED); - } + private void validateInput(OutputPointInfo request) throws BadItemException { if (request.getBlockNum() < 0 || request.getBlockNum() > 1000) { throw new BadItemException("request.BlockNum must be specified with range in [0, 1000]"); } @@ -2051,7 +2157,7 @@ private void validateInput(OutputPointInfo request) throws BadItemException, Zks public IncrementalMerkleVoucherInfo getMerkleTreeVoucherInfo(OutputPointInfo request) throws ItemNotFoundException, BadItemException, InvalidProtocolBufferException, ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); validateInput(request); IncrementalMerkleVoucherInfo.Builder result = IncrementalMerkleVoucherInfo.newBuilder(); @@ -2097,9 +2203,7 @@ public IncrementalMerkleVoucherInfo getMerkleTreeVoucherInfo(OutputPointInfo req } public IncrementalMerkleTree getMerkleTreeOfBlock(long blockNum) throws ZksnarkException { - if (!getFullNodeAllowShieldedTransaction()) { - throw new ZksnarkException(SHIELDED_ID_NOT_ALLOWED); - } + checkAllowShieldedTransactionApi(); if (blockNum < 0) { return null; } @@ -2166,7 +2270,7 @@ public ReceiveNote createReceiveNoteRandom(long value) throws ZksnarkException, public TransactionCapsule createShieldedTransaction(PrivateParameters request) throws ContractValidateException, RuntimeException, ZksnarkException, BadItemException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); ZenTransactionBuilder builder = new ZenTransactionBuilder(this); @@ -2268,7 +2372,7 @@ public TransactionCapsule createShieldedTransaction(PrivateParameters request) public TransactionCapsule createShieldedTransactionWithoutSpendAuthSig( PrivateParametersWithoutAsk request) throws ContractValidateException, ZksnarkException, BadItemException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); ZenTransactionBuilder builder = new ZenTransactionBuilder(this); @@ -2385,7 +2489,7 @@ private void shieldedOutput(List shieldedReceives, public ShieldedAddressInfo getNewShieldedAddress() throws BadItemException, ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); ShieldedAddressInfo.Builder addressInfo = ShieldedAddressInfo.newBuilder(); @@ -2418,7 +2522,7 @@ public ShieldedAddressInfo getNewShieldedAddress() throws BadItemException, Zksn } public BytesMessage getSpendingKey() throws ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); byte[] sk = SpendingKey.random().getValue(); return BytesMessage.newBuilder().setValue(ByteString.copyFrom(sk)).build(); @@ -2426,7 +2530,7 @@ public BytesMessage getSpendingKey() throws ZksnarkException { public ExpandedSpendingKeyMessage getExpandedSpendingKey(ByteString spendingKey) throws BadItemException, ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); if (Objects.isNull(spendingKey)) { throw new BadItemException("spendingKey is null"); @@ -2452,7 +2556,7 @@ public ExpandedSpendingKeyMessage getExpandedSpendingKey(ByteString spendingKey) public BytesMessage getAkFromAsk(ByteString ask) throws BadItemException, ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); if (Objects.isNull(ask)) { throw new BadItemException("ask is null"); @@ -2468,7 +2572,7 @@ public BytesMessage getAkFromAsk(ByteString ask) throws public BytesMessage getNkFromNsk(ByteString nsk) throws BadItemException, ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); if (Objects.isNull(nsk)) { throw new BadItemException("nsk is null"); @@ -2484,7 +2588,7 @@ public BytesMessage getNkFromNsk(ByteString nsk) throws public IncomingViewingKeyMessage getIncomingViewingKey(byte[] ak, byte[] nk) throws ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); byte[] ivk = new byte[32]; // the incoming viewing key JLibrustzcash.librustzcashCrhIvk(new CrhIvkParams(ak, nk, ivk)); @@ -2495,7 +2599,7 @@ public IncomingViewingKeyMessage getIncomingViewingKey(byte[] ak, byte[] nk) } public DiversifierMessage getDiversifier() throws ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); byte[] d; while (true) { @@ -2511,7 +2615,7 @@ public DiversifierMessage getDiversifier() throws ZksnarkException { } public BytesMessage getRcm() throws ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); byte[] rcm = Note.generateR(); return BytesMessage.newBuilder().setValue(ByteString.copyFrom(rcm)).build(); @@ -2519,7 +2623,7 @@ public BytesMessage getRcm() throws ZksnarkException { public PaymentAddressMessage getPaymentAddress(IncomingViewingKey ivk, DiversifierT d) throws BadItemException, ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); if (!JLibrustzcash.librustzcashCheckDiversifier(d.getData())) { throw new BadItemException("d is not valid"); @@ -2543,7 +2647,7 @@ public PaymentAddressMessage getPaymentAddress(IncomingViewingKey ivk, public SpendResult isSpend(NoteParameters noteParameters) throws ZksnarkException, InvalidProtocolBufferException, BadItemException, ItemNotFoundException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); GrpcAPI.Note note = noteParameters.getNote(); byte[] ak = noteParameters.getAk().toByteArray(); @@ -2599,7 +2703,7 @@ public SpendResult isSpend(NoteParameters noteParameters) throws public BytesMessage createSpendAuthSig(SpendAuthSigParameters spendAuthSigParameters) throws ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); byte[] result = new byte[64]; SpendSigParams spendSigParams = new SpendSigParams( @@ -2613,7 +2717,7 @@ public BytesMessage createSpendAuthSig(SpendAuthSigParameters spendAuthSigParame } public BytesMessage createShieldNullifier(NfParameters nfParameters) throws ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); byte[] ak = nfParameters.getAk().toByteArray(); byte[] nk = nfParameters.getNk().toByteArray(); @@ -2649,7 +2753,7 @@ public BytesMessage createShieldNullifier(NfParameters nfParameters) throws Zksn public BytesMessage getShieldTransactionHash(Transaction transaction) throws ContractValidateException, ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); List contract = transaction.getRawData().getContractList(); if (contract == null || contract.isEmpty()) { @@ -2671,37 +2775,7 @@ public BytesMessage getShieldTransactionHash(Transaction transaction) } public TransactionInfoList getTransactionInfoByBlockNum(long blockNum) { - TransactionInfoList.Builder transactionInfoList = TransactionInfoList.newBuilder(); - - try { - TransactionRetCapsule result = dbManager.getTransactionRetStore() - .getTransactionInfoByBlockNum(ByteArray.fromLong(blockNum)); - - if (!Objects.isNull(result) && !Objects.isNull(result.getInstance())) { - result.getInstance().getTransactioninfoList().forEach( - transactionInfo -> transactionInfoList.addTransactionInfo(transactionInfo) - ); - } else { - Block block = chainBaseManager.getBlockByNum(blockNum).getInstance(); - - if (block != null) { - List listTransaction = block.getTransactionsList(); - for (Transaction transaction : listTransaction) { - TransactionInfoCapsule transactionInfoCapsule = dbManager.getTransactionHistoryStore() - .get(Sha256Hash.hash(CommonParameter.getInstance() - .isECKeyCryptoEngine(), transaction.getRawData().toByteArray())); - - if (transactionInfoCapsule != null) { - transactionInfoList.addTransactionInfo(transactionInfoCapsule.getInstance()); - } - } - } - } - } catch (BadItemException | ItemNotFoundException e) { - logger.warn(e.getMessage()); - } - - return transactionInfoList.build(); + return dbManager.getTransactionInfoByBlockNum(blockNum); } public NodeList listNodes() { @@ -3354,7 +3428,7 @@ private GrpcAPI.DecryptNotes queryNoteByIvk(long startNum, long endNum, byte[] i */ public GrpcAPI.DecryptNotes scanNoteByIvk(long startNum, long endNum, byte[] ivk) throws BadItemException, ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); return queryNoteByIvk(startNum, endNum, ivk); } @@ -3365,7 +3439,7 @@ public GrpcAPI.DecryptNotes scanNoteByIvk(long startNum, long endNum, public GrpcAPI.DecryptNotesMarked scanAndMarkNoteByIvk(long startNum, long endNum, byte[] ivk, byte[] ak, byte[] nk) throws BadItemException, ZksnarkException, InvalidProtocolBufferException, ItemNotFoundException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); GrpcAPI.DecryptNotes srcNotes = queryNoteByIvk(startNum, endNum, ivk); GrpcAPI.DecryptNotesMarked.Builder builder = GrpcAPI.DecryptNotesMarked.newBuilder(); @@ -3398,7 +3472,7 @@ public GrpcAPI.DecryptNotesMarked scanAndMarkNoteByIvk(long startNum, long endNu */ public GrpcAPI.DecryptNotes scanNoteByOvk(long startNum, long endNum, byte[] ovk) throws BadItemException, ZksnarkException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); GrpcAPI.DecryptNotes.Builder builder = GrpcAPI.DecryptNotes.newBuilder(); if (!(startNum >= 0 && endNum > startNum && endNum - startNum <= 1000)) { @@ -3538,7 +3612,7 @@ private void buildShieldedTRC20Output(ShieldedTRC20ParametersBuilder builder, public ShieldedTRC20Parameters createShieldedContractParameters( PrivateShieldedTRC20Parameters request) throws ContractValidateException, ZksnarkException, ContractExeException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); ShieldedTRC20ParametersBuilder builder = new ShieldedTRC20ParametersBuilder(); @@ -3675,7 +3749,7 @@ private void buildShieldedTRC20InputWithAK( public ShieldedTRC20Parameters createShieldedContractParametersWithoutAsk( PrivateShieldedTRC20ParametersWithoutAsk request) throws ZksnarkException, ContractValidateException, ContractExeException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); ShieldedTRC20ParametersBuilder builder = new ShieldedTRC20ParametersBuilder(); byte[] shieldedTRC20ContractAddress = request.getShieldedTRC20ContractAddress().toByteArray(); @@ -3971,7 +4045,7 @@ public DecryptNotesTRC20 scanShieldedTRC20NotesByIvk( long startNum, long endNum, byte[] shieldedTRC20ContractAddress, byte[] ivk, byte[] ak, byte[] nk, ProtocolStringList topicsList) throws BadItemException, ZksnarkException, ContractExeException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); return queryTRC20NoteByIvk(startNum, endNum, shieldedTRC20ContractAddress, ivk, ak, nk, topicsList); @@ -4050,7 +4124,7 @@ private Optional getNoteTxFromLogListByOvk( public DecryptNotesTRC20 scanShieldedTRC20NotesByOvk(long startNum, long endNum, byte[] ovk, byte[] shieldedTRC20ContractAddress, ProtocolStringList topicsList) throws ZksnarkException, BadItemException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); DecryptNotesTRC20.Builder builder = DecryptNotesTRC20.newBuilder(); if (!(startNum >= 0 && endNum > startNum && endNum - startNum <= 1000)) { @@ -4113,7 +4187,7 @@ private byte[] getShieldedTRC20Nullifier(GrpcAPI.Note note, long pos, byte[] ak, public GrpcAPI.NullifierResult isShieldedTRC20ContractNoteSpent(NfTRC20Parameters request) throws ZksnarkException, ContractExeException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); return GrpcAPI.NullifierResult.newBuilder() .setIsSpent(isShieldedTRC20NoteSpent(request.getNote(), @@ -4236,7 +4310,7 @@ public byte[] getShieldedContractScalingFactor(byte[] contractAddress) public BytesMessage getTriggerInputForShieldedTRC20Contract( ShieldedTRC20TriggerContractParameters request) throws ZksnarkException, ContractValidateException { - checkFullNodeAllowShieldedTransaction(); + checkAllowShieldedTransactionApi(); ShieldedTRC20Parameters shieldedTRC20Parameters = request.getShieldedTRC20Parameters(); List spendAuthoritySignature = request.getSpendAuthoritySignatureList(); diff --git a/framework/src/main/java/org/tron/core/config/args/Args.java b/framework/src/main/java/org/tron/core/config/args/Args.java index 277b3e7bc79..46695986c1f 100644 --- a/framework/src/main/java/org/tron/core/config/args/Args.java +++ b/framework/src/main/java/org/tron/core/config/args/Args.java @@ -1,13 +1,18 @@ package org.tron.core.config.args; import static java.lang.System.exit; +import static org.fusesource.jansi.Ansi.ansi; import static org.tron.common.math.Maths.max; import static org.tron.common.math.Maths.min; import static org.tron.core.Constant.ADD_PRE_FIX_BYTE_MAINNET; +import static org.tron.core.Constant.DEFAULT_PROPOSAL_EXPIRE_TIME; import static org.tron.core.Constant.DYNAMIC_ENERGY_INCREASE_FACTOR_RANGE; import static org.tron.core.Constant.DYNAMIC_ENERGY_MAX_FACTOR_RANGE; +import static org.tron.core.Constant.MAX_PROPOSAL_EXPIRE_TIME; +import static org.tron.core.Constant.MIN_PROPOSAL_EXPIRE_TIME; import static org.tron.core.config.Parameter.ChainConstant.BLOCK_PRODUCE_TIMEOUT_PERCENT; import static org.tron.core.config.Parameter.ChainConstant.MAX_ACTIVE_WITNESS_NUM; +import static org.tron.core.exception.TronError.ErrCode.PARAMETER_INIT; import com.beust.jcommander.JCommander; import com.beust.jcommander.ParameterDescription; @@ -42,14 +47,15 @@ import lombok.extern.slf4j.Slf4j; import org.apache.commons.collections4.CollectionUtils; import org.apache.commons.lang3.StringUtils; +import org.fusesource.jansi.AnsiConsole; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; +import org.tron.common.arch.Arch; import org.tron.common.args.Account; import org.tron.common.args.GenesisBlock; import org.tron.common.args.Witness; import org.tron.common.config.DbBackupConfig; import org.tron.common.cron.CronExpression; -import org.tron.common.crypto.SignInterface; import org.tron.common.logsfilter.EventPluginConfig; import org.tron.common.logsfilter.FilterQuery; import org.tron.common.logsfilter.TriggerConfig; @@ -58,7 +64,6 @@ import org.tron.common.parameter.CommonParameter; import org.tron.common.parameter.RateLimiterInitialization; import org.tron.common.setting.RocksDbSettings; -import org.tron.common.utils.ByteArray; import org.tron.common.utils.Commons; import org.tron.common.utils.LocalWitnesses; import org.tron.core.Constant; @@ -66,11 +71,8 @@ import org.tron.core.config.Configuration; import org.tron.core.config.Parameter.NetConstants; import org.tron.core.config.Parameter.NodeConstant; -import org.tron.core.exception.CipherException; import org.tron.core.exception.TronError; import org.tron.core.store.AccountStore; -import org.tron.keystore.Credentials; -import org.tron.keystore.WalletUtils; import org.tron.p2p.P2pConfig; import org.tron.p2p.dns.update.DnsType; import org.tron.p2p.dns.update.PublishConfig; @@ -169,6 +171,7 @@ public static void clearParam() { PARAMETER.tcpNettyWorkThreadNum = 0; PARAMETER.udpNettyWorkThreadNum = 0; PARAMETER.solidityNode = false; + PARAMETER.keystoreFactory = false; PARAMETER.trustNodeAddr = ""; PARAMETER.walletExtensionApi = false; PARAMETER.estimateEnergy = false; @@ -186,7 +189,7 @@ public static void clearParam() { PARAMETER.maxHttpConnectNumber = 50; PARAMETER.allowMultiSign = 0; PARAMETER.trxExpirationTimeInMilliseconds = 0; - PARAMETER.fullNodeAllowShieldedTransactionArgs = true; + PARAMETER.allowShieldedTransactionApi = true; PARAMETER.zenTokenId = "000000"; PARAMETER.allowProtoFilterNum = 0; PARAMETER.allowAccountStateRoot = 0; @@ -204,6 +207,7 @@ public static void clearParam() { PARAMETER.jsonRpcHttpPBFTNodeEnable = false; PARAMETER.jsonRpcMaxBlockRange = 5000; PARAMETER.jsonRpcMaxSubTopics = 1000; + PARAMETER.jsonRpcMaxBlockFilterNum = 50000; PARAMETER.nodeMetricsEnable = false; PARAMETER.metricsStorageEnable = false; PARAMETER.metricsPrometheusEnable = false; @@ -235,6 +239,9 @@ public static void clearParam() { PARAMETER.rateLimiterGlobalQps = 50000; PARAMETER.rateLimiterGlobalIpQps = 10000; PARAMETER.rateLimiterGlobalApiQps = 1000; + PARAMETER.rateLimiterSyncBlockChain = 3.0; + PARAMETER.rateLimiterFetchInvData = 3.0; + PARAMETER.rateLimiterDisconnect = 1.0; PARAMETER.p2pDisable = false; PARAMETER.dynamicConfigEnable = false; PARAMETER.dynamicConfigCheckInterval = 600; @@ -247,6 +254,8 @@ public static void clearParam() { PARAMETER.consensusLogicOptimization = 0; PARAMETER.allowTvmCancun = 0; PARAMETER.allowTvmBlob = 0; + PARAMETER.rpcMaxRstStream = 0; + PARAMETER.rpcSecondsPerWindow = 0; } /** @@ -342,7 +351,7 @@ private static String getCommitIdAbbrev() { private static Map getOptionGroup() { String[] tronOption = new String[] {"version", "help", "shellConfFileName", "logbackPath", - "eventSubscribe"}; + "eventSubscribe", "solidityNode", "keystoreFactory"}; String[] dbOption = new String[] {"outputDirectory"}; String[] witnessOption = new String[] {"witness", "privateKey"}; String[] vmOption = new String[] {"debug"}; @@ -369,6 +378,17 @@ private static Map getOptionGroup() { * set parameters. */ public static void setParam(final String[] args, final String confFileName) { + try { + Arch.throwIfUnsupportedJavaVersion(); + } catch (UnsupportedOperationException e) { + AnsiConsole.systemInstall(); + // To avoid confusion caused by silent execution when using -h or -v flags, + // errors are explicitly logged to the console in this context. + // Console output is not required for errors in other scenarios. + System.out.println(ansi().fgRed().a(e.getMessage()).reset()); + AnsiConsole.systemUninstall(); + throw new TronError(e, TronError.ErrCode.JDK_VERSION); + } JCommander.newBuilder().addObject(PARAMETER).build().parse(args); if (PARAMETER.version) { printVersion(); @@ -403,61 +423,7 @@ public static void setParam(final Config config) { PARAMETER.cryptoEngine = config.hasPath(Constant.CRYPTO_ENGINE) ? config .getString(Constant.CRYPTO_ENGINE) : Constant.ECKey_ENGINE; - if (StringUtils.isNoneBlank(PARAMETER.privateKey)) { - localWitnesses = (new LocalWitnesses(PARAMETER.privateKey)); - if (StringUtils.isNoneBlank(PARAMETER.witnessAddress)) { - byte[] bytes = Commons.decodeFromBase58Check(PARAMETER.witnessAddress); - if (bytes != null) { - localWitnesses.setWitnessAccountAddress(bytes); - logger.debug("Got localWitnessAccountAddress from cmd"); - } else { - PARAMETER.witnessAddress = ""; - logger.warn(IGNORE_WRONG_WITNESS_ADDRESS_FORMAT); - } - } - localWitnesses.initWitnessAccountAddress(PARAMETER.isECKeyCryptoEngine()); - logger.debug("Got privateKey from cmd"); - } else if (config.hasPath(Constant.LOCAL_WITNESS)) { - localWitnesses = new LocalWitnesses(); - List localwitness = config.getStringList(Constant.LOCAL_WITNESS); - localWitnesses.setPrivateKeys(localwitness); - witnessAddressCheck(config); - localWitnesses.initWitnessAccountAddress(PARAMETER.isECKeyCryptoEngine()); - logger.debug("Got privateKey from config.conf"); - } else if (config.hasPath(Constant.LOCAL_WITNESS_KEYSTORE)) { - localWitnesses = new LocalWitnesses(); - List privateKeys = new ArrayList(); - if (PARAMETER.isWitness()) { - List localwitness = config.getStringList(Constant.LOCAL_WITNESS_KEYSTORE); - if (localwitness.size() > 0) { - String fileName = System.getProperty("user.dir") + "/" + localwitness.get(0); - String password; - if (StringUtils.isEmpty(PARAMETER.password)) { - System.out.println("Please input your password."); - password = WalletUtils.inputPassword(); - } else { - password = PARAMETER.password; - PARAMETER.password = null; - } - - try { - Credentials credentials = WalletUtils - .loadCredentials(password, new File(fileName)); - SignInterface sign = credentials.getSignInterface(); - String prikey = ByteArray.toHexString(sign.getPrivateKey()); - privateKeys.add(prikey); - } catch (IOException | CipherException e) { - logger.error("Witness node start failed!"); - throw new TronError(e, TronError.ErrCode.WITNESS_KEYSTORE_LOAD); - } - } - } - localWitnesses.setPrivateKeys(privateKeys); - witnessAddressCheck(config); - localWitnesses.initWitnessAccountAddress(PARAMETER.isECKeyCryptoEngine()); - logger.debug("Got privateKey from keystore"); - } - + localWitnesses = new WitnessInitializer(config).initLocalWitnesses(); if (PARAMETER.isWitness() && CollectionUtils.isEmpty(localWitnesses.getPrivateKeys())) { throw new TronError("This is a witness node, but localWitnesses is null", @@ -526,6 +492,11 @@ public static void setParam(final Config config) { config.getInt(Constant.NODE_JSONRPC_MAX_SUB_TOPICS); } + if (config.hasPath(Constant.NODE_JSONRPC_MAX_BLOCK_FILTER_NUM)) { + PARAMETER.jsonRpcMaxBlockFilterNum = + config.getInt(Constant.NODE_JSONRPC_MAX_BLOCK_FILTER_NUM); + } + if (config.hasPath(Constant.VM_MIN_TIME_RATIO)) { PARAMETER.minTimeRatio = config.getDouble(Constant.VM_MIN_TIME_RATIO); } @@ -771,6 +742,12 @@ public static void setParam(final Config config) { PARAMETER.flowControlWindow = config.hasPath(Constant.NODE_RPC_FLOW_CONTROL_WINDOW) ? config.getInt(Constant.NODE_RPC_FLOW_CONTROL_WINDOW) : NettyServerBuilder.DEFAULT_FLOW_CONTROL_WINDOW; + if (config.hasPath(Constant.NODE_RPC_MAX_RST_STREAM)) { + PARAMETER.rpcMaxRstStream = config.getInt(Constant.NODE_RPC_MAX_RST_STREAM); + } + if (config.hasPath(Constant.NODE_RPC_SECONDS_PER_WINDOW)) { + PARAMETER.rpcSecondsPerWindow = config.getInt(Constant.NODE_RPC_SECONDS_PER_WINDOW); + } PARAMETER.maxConnectionIdleInMillis = config.hasPath(Constant.NODE_RPC_MAX_CONNECTION_IDLE_IN_MILLIS) @@ -815,9 +792,7 @@ public static void setParam(final Config config) { config.hasPath(Constant.BLOCK_MAINTENANCE_TIME_INTERVAL) ? config .getInt(Constant.BLOCK_MAINTENANCE_TIME_INTERVAL) : 21600000L; - PARAMETER.proposalExpireTime = - config.hasPath(Constant.BLOCK_PROPOSAL_EXPIRE_TIME) ? config - .getInt(Constant.BLOCK_PROPOSAL_EXPIRE_TIME) : 259200000L; + PARAMETER.proposalExpireTime = getProposalExpirationTime(config); PARAMETER.checkFrozenTime = config.hasPath(Constant.BLOCK_CHECK_FROZEN_TIME) ? config @@ -991,9 +966,18 @@ public static void setParam(final Config config) { PARAMETER.eventFilter = config.hasPath(Constant.EVENT_SUBSCRIBE_FILTER) ? getEventFilter(config) : null; - PARAMETER.fullNodeAllowShieldedTransactionArgs = - !config.hasPath(Constant.NODE_FULLNODE_ALLOW_SHIELDED_TRANSACTION) - || config.getBoolean(Constant.NODE_FULLNODE_ALLOW_SHIELDED_TRANSACTION); + if (config.hasPath(Constant.ALLOW_SHIELDED_TRANSACTION_API)) { + PARAMETER.allowShieldedTransactionApi = + config.getBoolean(Constant.ALLOW_SHIELDED_TRANSACTION_API); + } else if (config.hasPath(Constant.NODE_FULLNODE_ALLOW_SHIELDED_TRANSACTION)) { + // for compatibility with previous configuration + PARAMETER.allowShieldedTransactionApi = + config.getBoolean(Constant.NODE_FULLNODE_ALLOW_SHIELDED_TRANSACTION); + logger.warn("Configuring [node.fullNodeAllowShieldedTransaction] will be deprecated. " + + "Please use [node.allowShieldedTransactionApi] instead."); + } else { + PARAMETER.allowShieldedTransactionApi = true; + } PARAMETER.zenTokenId = config.hasPath(Constant.NODE_ZEN_TOKENID) ? config.getString(Constant.NODE_ZEN_TOKENID) : "000000"; @@ -1030,10 +1014,6 @@ public static void setParam(final Config config) { config.hasPath(Constant.NODE_SHIELDED_TRANS_IN_PENDING_MAX_COUNTS) ? config .getInt(Constant.NODE_SHIELDED_TRANS_IN_PENDING_MAX_COUNTS) : 10; - if (PARAMETER.isWitness()) { - PARAMETER.fullNodeAllowShieldedTransactionArgs = true; - } - PARAMETER.rateLimiterGlobalQps = config.hasPath(Constant.RATE_LIMITER_GLOBAL_QPS) ? config .getInt(Constant.RATE_LIMITER_GLOBAL_QPS) : 50000; @@ -1048,6 +1028,18 @@ public static void setParam(final Config config) { PARAMETER.rateLimiterInitialization = getRateLimiterFromConfig(config); + PARAMETER.rateLimiterSyncBlockChain = + config.hasPath(Constant.RATE_LIMITER_P2P_SYNC_BLOCK_CHAIN) ? config + .getDouble(Constant.RATE_LIMITER_P2P_SYNC_BLOCK_CHAIN) : 3.0; + + PARAMETER.rateLimiterFetchInvData = + config.hasPath(Constant.RATE_LIMITER_P2P_FETCH_INV_DATA) ? config + .getDouble(Constant.RATE_LIMITER_P2P_FETCH_INV_DATA) : 3.0; + + PARAMETER.rateLimiterDisconnect = + config.hasPath(Constant.RATE_LIMITER_P2P_DISCONNECT) ? config + .getDouble(Constant.RATE_LIMITER_P2P_DISCONNECT) : 1.0; + PARAMETER.changedDelegation = config.hasPath(Constant.COMMITTEE_CHANGED_DELEGATION) ? config .getInt(Constant.COMMITTEE_CHANGED_DELEGATION) : 0; @@ -1300,6 +1292,25 @@ public static void setParam(final Config config) { logConfig(); } + private static long getProposalExpirationTime(final Config config) { + if (config.hasPath(Constant.COMMITTEE_PROPOSAL_EXPIRE_TIME)) { + throw new TronError("It is not allowed to configure committee.proposalExpireTime in " + + "config.conf, please set the value in block.proposalExpireTime.", PARAMETER_INIT); + } + if (config.hasPath(Constant.BLOCK_PROPOSAL_EXPIRE_TIME)) { + long proposalExpireTime = config.getLong(Constant.BLOCK_PROPOSAL_EXPIRE_TIME); + if (proposalExpireTime <= MIN_PROPOSAL_EXPIRE_TIME + || proposalExpireTime >= MAX_PROPOSAL_EXPIRE_TIME) { + throw new TronError("The value[block.proposalExpireTime] is only allowed to " + + "be greater than " + MIN_PROPOSAL_EXPIRE_TIME + " and less than " + + MAX_PROPOSAL_EXPIRE_TIME + "!", PARAMETER_INIT); + } + return proposalExpireTime; + } else { + return DEFAULT_PROPOSAL_EXPIRE_TIME; + } + } + private static List getWitnessesFromConfig(final com.typesafe.config.Config config) { return config.getObjectList(Constant.GENESIS_BLOCK_WITNESSES).stream() .map(Args::createWitness) @@ -1622,7 +1633,7 @@ private static FilterQuery getEventFilter(final com.typesafe.config.Config confi try { fromBlockLong = FilterQuery.parseFromBlockNumber(fromBlock); } catch (Exception e) { - logger.error("{}", e); + logger.error("invalid filter: fromBlockNumber: {}", fromBlock, e); return null; } filter.setFromBlock(fromBlockLong); @@ -1631,7 +1642,7 @@ private static FilterQuery getEventFilter(final com.typesafe.config.Config confi try { toBlockLong = FilterQuery.parseToBlockNumber(toBlock); } catch (Exception e) { - logger.error("{}", e); + logger.error("invalid filter: toBlockNumber: {}", toBlock, e); return null; } filter.setToBlock(toBlockLong); @@ -1684,11 +1695,13 @@ private static void initRocksDbSettings(Config config) { .getLong(prefix + "targetFileSizeBase") : 64; int targetFileSizeMultiplier = config.hasPath(prefix + "targetFileSizeMultiplier") ? config .getInt(prefix + "targetFileSizeMultiplier") : 1; + int maxOpenFiles = config.hasPath(prefix + "maxOpenFiles") + ? config.getInt(prefix + "maxOpenFiles") : 5000; PARAMETER.rocksDBCustomSettings = RocksDbSettings .initCustomSettings(levelNumber, compactThreads, blocksize, maxBytesForLevelBase, maxBytesForLevelMultiplier, level0FileNumCompactionTrigger, - targetFileSizeBase, targetFileSizeMultiplier); + targetFileSizeBase, targetFileSizeMultiplier, maxOpenFiles); RocksDbSettings.loggingSettings(); } @@ -1725,6 +1738,8 @@ private static void initBackupProperty(Config config) { public static void logConfig() { CommonParameter parameter = CommonParameter.getInstance(); logger.info("\n"); + logger.info("************************ System info ************************"); + logger.info("{}", Arch.withAll()); logger.info("************************ Net config ************************"); logger.info("P2P version: {}", parameter.getNodeP2pVersion()); logger.info("LAN IP: {}", parameter.getNodeLanIp()); @@ -1769,23 +1784,6 @@ public static void logConfig() { logger.info("\n"); } - public static void setFullNodeAllowShieldedTransaction(boolean fullNodeAllowShieldedTransaction) { - PARAMETER.fullNodeAllowShieldedTransactionArgs = fullNodeAllowShieldedTransaction; - } - - private static void witnessAddressCheck(Config config) { - if (config.hasPath(Constant.LOCAL_WITNESS_ACCOUNT_ADDRESS)) { - byte[] bytes = Commons - .decodeFromBase58Check(config.getString(Constant.LOCAL_WITNESS_ACCOUNT_ADDRESS)); - if (bytes != null) { - localWitnesses.setWitnessAccountAddress(bytes); - logger.debug("Got localWitnessAccountAddress from config.conf"); - } else { - logger.warn(IGNORE_WRONG_WITNESS_ADDRESS_FORMAT); - } - } - } - /** * get output directory. */ diff --git a/framework/src/main/java/org/tron/core/config/args/WitnessInitializer.java b/framework/src/main/java/org/tron/core/config/args/WitnessInitializer.java new file mode 100644 index 00000000000..2ea3a449ef4 --- /dev/null +++ b/framework/src/main/java/org/tron/core/config/args/WitnessInitializer.java @@ -0,0 +1,149 @@ +package org.tron.core.config.args; + +import com.typesafe.config.Config; +import java.io.File; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import lombok.extern.slf4j.Slf4j; +import org.apache.commons.lang3.StringUtils; +import org.tron.common.crypto.SignInterface; +import org.tron.common.utils.ByteArray; +import org.tron.common.utils.Commons; +import org.tron.common.utils.LocalWitnesses; +import org.tron.core.Constant; +import org.tron.core.exception.CipherException; +import org.tron.core.exception.TronError; +import org.tron.keystore.Credentials; +import org.tron.keystore.WalletUtils; + +@Slf4j +public class WitnessInitializer { + + private final Config config; + + private LocalWitnesses localWitnesses; + + public WitnessInitializer(Config config) { + this.config = config; + this.localWitnesses = new LocalWitnesses(); + } + + public LocalWitnesses initLocalWitnesses() { + if (!Args.PARAMETER.isWitness()) { + return localWitnesses; + } + + if (tryInitFromCommandLine()) { + return localWitnesses; + } + + if (tryInitFromConfig()) { + return localWitnesses; + } + + tryInitFromKeystore(); + + return localWitnesses; + } + + private boolean tryInitFromCommandLine() { + if (StringUtils.isBlank(Args.PARAMETER.privateKey)) { + return false; + } + + byte[] witnessAddress = null; + this.localWitnesses = new LocalWitnesses(Args.PARAMETER.privateKey); + if (StringUtils.isNotEmpty(Args.PARAMETER.witnessAddress)) { + witnessAddress = Commons.decodeFromBase58Check(Args.PARAMETER.witnessAddress); + if (witnessAddress == null) { + throw new TronError("LocalWitnessAccountAddress format from cmd is incorrect", + TronError.ErrCode.WITNESS_INIT); + } + logger.debug("Got localWitnessAccountAddress from cmd"); + } + + this.localWitnesses.initWitnessAccountAddress(witnessAddress, + Args.PARAMETER.isECKeyCryptoEngine()); + logger.debug("Got privateKey from cmd"); + return true; + } + + private boolean tryInitFromConfig() { + if (!config.hasPath(Constant.LOCAL_WITNESS) || config.getStringList(Constant.LOCAL_WITNESS) + .isEmpty()) { + return false; + } + + List localWitness = config.getStringList(Constant.LOCAL_WITNESS); + this.localWitnesses.setPrivateKeys(localWitness); + logger.debug("Got privateKey from config.conf"); + byte[] witnessAddress = getWitnessAddress(); + this.localWitnesses.initWitnessAccountAddress(witnessAddress, + Args.PARAMETER.isECKeyCryptoEngine()); + return true; + } + + private void tryInitFromKeystore() { + if (!config.hasPath(Constant.LOCAL_WITNESS_KEYSTORE) + || config.getStringList(Constant.LOCAL_WITNESS_KEYSTORE).isEmpty()) { + return; + } + + List localWitness = config.getStringList(Constant.LOCAL_WITNESS_KEYSTORE); + if (localWitness.size() > 1) { + logger.warn( + "Multiple keystores detected. Only the first keystore will be used as witness, all " + + "others will be ignored."); + } + + List privateKeys = new ArrayList<>(); + String fileName = System.getProperty("user.dir") + "/" + localWitness.get(0); + String password; + if (StringUtils.isEmpty(Args.PARAMETER.password)) { + System.out.println("Please input your password."); + password = WalletUtils.inputPassword(); + } else { + password = Args.PARAMETER.password; + Args.PARAMETER.password = null; + } + + try { + Credentials credentials = WalletUtils + .loadCredentials(password, new File(fileName)); + SignInterface sign = credentials.getSignInterface(); + String prikey = ByteArray.toHexString(sign.getPrivateKey()); + privateKeys.add(prikey); + } catch (IOException | CipherException e) { + logger.error("Witness node start failed!"); + throw new TronError(e, TronError.ErrCode.WITNESS_KEYSTORE_LOAD); + } + + this.localWitnesses.setPrivateKeys(privateKeys); + byte[] witnessAddress = getWitnessAddress(); + this.localWitnesses.initWitnessAccountAddress(witnessAddress, + Args.PARAMETER.isECKeyCryptoEngine()); + logger.debug("Got privateKey from keystore"); + } + + private byte[] getWitnessAddress() { + if (!config.hasPath(Constant.LOCAL_WITNESS_ACCOUNT_ADDRESS)) { + return null; + } + + if (localWitnesses.getPrivateKeys().size() != 1) { + throw new TronError( + "LocalWitnessAccountAddress can only be set when there is only one private key", + TronError.ErrCode.WITNESS_INIT); + } + byte[] witnessAddress = Commons + .decodeFromBase58Check(config.getString(Constant.LOCAL_WITNESS_ACCOUNT_ADDRESS)); + if (witnessAddress != null) { + logger.debug("Got localWitnessAccountAddress from config.conf"); + } else { + throw new TronError("LocalWitnessAccountAddress format from config is incorrect", + TronError.ErrCode.WITNESS_INIT); + } + return witnessAddress; + } +} diff --git a/framework/src/main/java/org/tron/core/consensus/ConsensusService.java b/framework/src/main/java/org/tron/core/consensus/ConsensusService.java index ce1f1f1cf08..ef8f30ef498 100644 --- a/framework/src/main/java/org/tron/core/consensus/ConsensusService.java +++ b/framework/src/main/java/org/tron/core/consensus/ConsensusService.java @@ -61,17 +61,18 @@ public void start() { logger.info("Add witness: {}, size: {}", Hex.toHexString(privateKeyAddress), miners.size()); } - } else { + } else if (privateKeys.size() == 1) { byte[] privateKey = fromHexString(Args.getLocalWitnesses().getPrivateKey()); byte[] privateKeyAddress = SignUtils.fromPrivate(privateKey, Args.getInstance().isECKeyCryptoEngine()).getAddress(); - byte[] witnessAddress = Args.getLocalWitnesses().getWitnessAccountAddress( - Args.getInstance().isECKeyCryptoEngine()); + byte[] witnessAddress = Args.getLocalWitnesses().getWitnessAccountAddress(); WitnessCapsule witnessCapsule = witnessStore.get(witnessAddress); if (null == witnessCapsule) { logger.warn("Witness {} is not in witnessStore.", Hex.toHexString(witnessAddress)); } + // In multi-signature mode, the address derived from the private key may differ from + // witnessAddress. Miner miner = param.new Miner(privateKey, ByteString.copyFrom(privateKeyAddress), ByteString.copyFrom(witnessAddress)); miners.add(miner); diff --git a/framework/src/main/java/org/tron/core/consensus/ProposalService.java b/framework/src/main/java/org/tron/core/consensus/ProposalService.java index b25f0d6fa8d..51d53f6a59e 100644 --- a/framework/src/main/java/org/tron/core/consensus/ProposalService.java +++ b/framework/src/main/java/org/tron/core/consensus/ProposalService.java @@ -384,6 +384,14 @@ public static boolean process(Manager manager, ProposalCapsule proposalCapsule) manager.getDynamicPropertiesStore().saveAllowTvmBlob(entry.getValue()); break; } + case ALLOW_TVM_SELFDESTRUCT_RESTRICTION: { + manager.getDynamicPropertiesStore().saveAllowTvmSelfdestructRestriction(entry.getValue()); + break; + } + case PROPOSAL_EXPIRE_TIME: { + manager.getDynamicPropertiesStore().saveProposalExpireTime(entry.getValue()); + break; + } default: find = false; break; diff --git a/framework/src/main/java/org/tron/core/db/Manager.java b/framework/src/main/java/org/tron/core/db/Manager.java index 1eecc103874..cd1a61c01fe 100644 --- a/framework/src/main/java/org/tron/core/db/Manager.java +++ b/framework/src/main/java/org/tron/core/db/Manager.java @@ -21,6 +21,7 @@ import java.util.Collections; import java.util.Comparator; import java.util.Date; +import java.util.HashMap; import java.util.HashSet; import java.util.LinkedList; import java.util.List; @@ -48,6 +49,7 @@ import org.bouncycastle.util.encoders.Hex; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; +import org.tron.api.GrpcAPI; import org.tron.api.GrpcAPI.TransactionInfoList; import org.tron.common.args.GenesisBlock; import org.tron.common.bloom.Bloom; @@ -97,6 +99,7 @@ import org.tron.core.capsule.TransactionRetCapsule; import org.tron.core.capsule.WitnessCapsule; import org.tron.core.capsule.utils.TransactionUtil; +import org.tron.core.config.Parameter; import org.tron.core.config.Parameter.ChainConstant; import org.tron.core.config.args.Args; import org.tron.core.consensus.ProposalController; @@ -163,6 +166,7 @@ import org.tron.core.store.WitnessScheduleStore; import org.tron.core.store.WitnessStore; import org.tron.core.utils.TransactionRegister; +import org.tron.protos.Protocol; import org.tron.protos.Protocol.AccountType; import org.tron.protos.Protocol.Permission; import org.tron.protos.Protocol.Transaction; @@ -868,9 +872,13 @@ public boolean pushTransaction(final TransactionCapsule trx) TooBigTransactionException, TransactionExpirationException, ReceiptCheckErrException, VMIllegalException, TooBigTransactionResultException { - if (isShieldedTransaction(trx.getInstance()) && !Args.getInstance() - .isFullNodeAllowShieldedTransactionArgs()) { - return true; + if (isShieldedTransaction(trx.getInstance()) && !chainBaseManager.getDynamicPropertiesStore() + .supportShieldedTransaction()) { + throw new ContractValidateException("ShieldedTransferContract is not supported."); + } + + if (isExchangeTransaction(trx.getInstance())) { + throw new ContractValidateException("ExchangeTransactionContract is rejected"); } pushTransactionQueue.add(trx); @@ -1192,6 +1200,28 @@ private void switchFork(BlockCapsule newHead) } + private boolean isSameSig(TransactionCapsule tx1, TransactionCapsule tx2) { + if (tx1 == null || tx2 == null) { + return false; + } + + if (tx1.getInstance().getSignatureCount() != tx2.getInstance().getSignatureCount()) { + return false; + } + + boolean flag = true; + for (int i = 0; i < tx1.getInstance().getSignatureCount(); i++) { + ByteString sig1 = tx1.getInstance().getSignature(i); + ByteString sig2 = tx2.getInstance().getSignature(i); + if (!sig1.equals(sig2)) { + flag = false; + break; + } + } + + return flag; + } + public List getVerifyTxs(BlockCapsule block) { if (pendingTransactions.size() == 0) { @@ -1199,7 +1229,7 @@ public List getVerifyTxs(BlockCapsule block) { } List txs = new ArrayList<>(); - Set txIds = new HashSet<>(); + Map txMap = new HashMap<>(); Set multiAddresses = new HashSet<>(); pendingTransactions.forEach(capsule -> { @@ -1208,14 +1238,14 @@ public List getVerifyTxs(BlockCapsule block) { String address = Hex.toHexString(capsule.getOwnerAddress()); multiAddresses.add(address); } else { - txIds.add(txId); + txMap.put(txId, capsule); } }); block.getTransactions().forEach(capsule -> { String address = Hex.toHexString(capsule.getOwnerAddress()); String txId = Hex.toHexString(capsule.getTransactionId().getBytes()); - if (multiAddresses.contains(address) || !txIds.contains(txId)) { + if (multiAddresses.contains(address) || !isSameSig(capsule, txMap.get(txId))) { txs.add(capsule); } else { capsule.setVerified(true); @@ -1676,6 +1706,11 @@ public BlockCapsule generateBlock(Miner miner, long blockTime, long timeout) { accountSet.add(ownerAddress); } } + + if (isExchangeTransaction(transaction)) { + continue; + } + if (ownerAddressSet.contains(ownerAddress)) { trx.setVerified(false); } @@ -1749,6 +1784,24 @@ private boolean isShieldedTransaction(Transaction transaction) { } } + private boolean isExchangeTransaction(Transaction transaction) { + Contract contract = transaction.getRawData().getContract(0); + switch (contract.getType()) { + case ExchangeTransactionContract: { + return true; + } + default: + return false; + } + } + + private void rejectExchangeTransaction(Transaction transaction) throws ContractValidateException { + if (isExchangeTransaction(transaction) && chainBaseManager.getForkController() + .pass(Parameter.ForkBlockVersionEnum.VERSION_4_8_0_1)) { + throw new ContractValidateException("ExchangeTransactionContract is rejected"); + } + } + public TransactionStore getTransactionStore() { return chainBaseManager.getTransactionStore(); } @@ -1803,6 +1856,7 @@ private void processBlock(BlockCapsule block, List txs) List results = new ArrayList<>(); long num = block.getNum(); for (TransactionCapsule transactionCapsule : block.getTransactions()) { + rejectExchangeTransaction(transactionCapsule.getInstance()); if (chainBaseManager.getDynamicPropertiesStore().allowConsensusLogicOptimization() && transactionCapsule.retCountIsGreatThanContractCount()) { throw new BadBlockException(String.format("The result count %d of this transaction %s is " @@ -1857,12 +1911,10 @@ private void processBlock(BlockCapsule block, List txs) chainBaseManager.getBalanceTraceStore().resetCurrentBlockTrace(); - if (CommonParameter.getInstance().isJsonRpcFilterEnabled()) { - Bloom blockBloom = chainBaseManager.getSectionBloomStore() - .initBlockSection(transactionRetCapsule); - chainBaseManager.getSectionBloomStore().write(block.getNum()); - block.setBloom(blockBloom); - } + Bloom blockBloom = chainBaseManager.getSectionBloomStore() + .initBlockSection(transactionRetCapsule); + chainBaseManager.getSectionBloomStore().write(block.getNum()); + block.setBloom(blockBloom); } private void payReward(BlockCapsule block) { @@ -2159,25 +2211,7 @@ private void processTransactionTrigger(BlockCapsule newBlock) { // need to set eth compatible data from transactionInfoList if (EventPluginLoader.getInstance().isTransactionLogTriggerEthCompatible() && newBlock.getNum() != 0) { - TransactionInfoList transactionInfoList = TransactionInfoList.newBuilder().build(); - TransactionInfoList.Builder transactionInfoListBuilder = TransactionInfoList.newBuilder(); - - try { - TransactionRetCapsule result = chainBaseManager.getTransactionRetStore() - .getTransactionInfoByBlockNum(ByteArray.fromLong(newBlock.getNum())); - - if (!Objects.isNull(result) && !Objects.isNull(result.getInstance())) { - result.getInstance().getTransactioninfoList().forEach( - transactionInfoListBuilder::addTransactionInfo - ); - - transactionInfoList = transactionInfoListBuilder.build(); - } - } catch (BadItemException e) { - logger.error("PostBlockTrigger getTransactionInfoList blockNum = {}, error is {}.", - newBlock.getNum(), e.getMessage()); - } - + TransactionInfoList transactionInfoList = getTransactionInfoByBlockNum(newBlock.getNum()); if (transactionCapsuleList.size() == transactionInfoList.getTransactionInfoCount()) { long cumulativeEnergyUsed = 0; long cumulativeLogCount = 0; @@ -2235,21 +2269,8 @@ private void postLogsFilter(final BlockCapsule blockCapsule, boolean solidified, boolean removed) { if (!blockCapsule.getTransactions().isEmpty()) { long blockNumber = blockCapsule.getNum(); - List transactionInfoList = new ArrayList<>(); - - try { - TransactionRetCapsule result = chainBaseManager.getTransactionRetStore() - .getTransactionInfoByBlockNum(ByteArray.fromLong(blockNumber)); - - if (!Objects.isNull(result) && !Objects.isNull(result.getInstance())) { - transactionInfoList.addAll(result.getInstance().getTransactioninfoList()); - } - } catch (BadItemException e) { - logger.error("ProcessLogsFilter getTransactionInfoList blockNum = {}, error is {}.", - blockNumber, e.getMessage()); - return; - } - + List transactionInfoList + = getTransactionInfoByBlockNum(blockNumber).getTransactionInfoList(); LogsFilterCapsule logsFilterCapsule = new LogsFilterCapsule(blockNumber, blockCapsule.getBlockId().toString(), blockCapsule.getBloom(), transactionInfoList, solidified, removed); @@ -2490,6 +2511,40 @@ private boolean isBlockWaitingLock() { return blockWaitLock.get() > NO_BLOCK_WAITING_LOCK; } + public TransactionInfoList getTransactionInfoByBlockNum(long blockNum) { + TransactionInfoList.Builder transactionInfoList = TransactionInfoList.newBuilder(); + + try { + TransactionRetCapsule result = getTransactionRetStore() + .getTransactionInfoByBlockNum(ByteArray.fromLong(blockNum)); + + if (!Objects.isNull(result) && !Objects.isNull(result.getInstance())) { + result.getInstance().getTransactioninfoList().forEach( + transactionInfo -> transactionInfoList.addTransactionInfo(transactionInfo) + ); + } else { + Protocol.Block block = chainBaseManager.getBlockByNum(blockNum).getInstance(); + + if (block != null) { + List listTransaction = block.getTransactionsList(); + for (Transaction transaction : listTransaction) { + TransactionInfoCapsule transactionInfoCapsule = getTransactionHistoryStore() + .get(Sha256Hash.hash(CommonParameter.getInstance() + .isECKeyCryptoEngine(), transaction.getRawData().toByteArray())); + + if (transactionInfoCapsule != null) { + transactionInfoList.addTransactionInfo(transactionInfoCapsule.getInstance()); + } + } + } + } + } catch (BadItemException | ItemNotFoundException e) { + logger.warn(e.getMessage()); + } + + return transactionInfoList.build(); + } + public void close() { stopRePushThread(); stopRePushTriggerThread(); diff --git a/framework/src/main/java/org/tron/core/metrics/node/NodeMetricManager.java b/framework/src/main/java/org/tron/core/metrics/node/NodeMetricManager.java index a0aba129648..9a5ecb33213 100644 --- a/framework/src/main/java/org/tron/core/metrics/node/NodeMetricManager.java +++ b/framework/src/main/java/org/tron/core/metrics/node/NodeMetricManager.java @@ -4,7 +4,7 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; import org.tron.common.backup.BackupManager; -import org.tron.common.parameter.CommonParameter; +import org.tron.common.utils.ByteArray; import org.tron.core.ChainBaseManager; import org.tron.core.config.args.Args; import org.tron.program.Version; @@ -36,8 +36,9 @@ private void setNodeInfo(NodeInfo nodeInfo) { nodeInfo.setIp(Args.getInstance().getNodeExternalIp()); - ByteString witnessAddress = ByteString.copyFrom(Args.getLocalWitnesses() - .getWitnessAccountAddress(CommonParameter.getInstance().isECKeyCryptoEngine())); + byte[] witnessAccountAddress = Args.getLocalWitnesses().getWitnessAccountAddress(); + ByteString witnessAddress = !ByteArray.isEmpty(witnessAccountAddress) ? ByteString + .copyFrom(witnessAccountAddress) : null; if (chainBaseManager.getWitnessScheduleStore().getActiveWitnesses().contains(witnessAddress)) { nodeInfo.setNodeType(1); } else { diff --git a/framework/src/main/java/org/tron/core/net/P2pEventHandlerImpl.java b/framework/src/main/java/org/tron/core/net/P2pEventHandlerImpl.java index 795c90b4edd..9cfa5058e8c 100644 --- a/framework/src/main/java/org/tron/core/net/P2pEventHandlerImpl.java +++ b/framework/src/main/java/org/tron/core/net/P2pEventHandlerImpl.java @@ -178,9 +178,11 @@ private void processMessage(PeerConnection peer, byte[] data) { handshakeService.processHelloMessage(peer, (HelloMessage) msg); break; case P2P_DISCONNECT: - peer.getChannel().close(); - peer.getNodeStatistics() - .nodeDisconnectedRemote(((DisconnectMessage)msg).getReason()); + if (peer.getP2pRateLimiter().tryAcquire(type.asByte())) { + peer.getChannel().close(); + peer.getNodeStatistics() + .nodeDisconnectedRemote(((DisconnectMessage)msg).getReason()); + } break; case SYNC_BLOCK_CHAIN: syncBlockChainMsgHandler.processMessage(peer, msg); @@ -253,12 +255,14 @@ private void processException(PeerConnection peer, TronMessage msg, Exception ex code = Protocol.ReasonCode.BAD_TX; break; case BAD_BLOCK: + case BLOCK_SIGN_ERROR: code = Protocol.ReasonCode.BAD_BLOCK; break; case NO_SUCH_MESSAGE: code = Protocol.ReasonCode.NO_SUCH_MESSAGE; break; case BAD_MESSAGE: + case RATE_LIMIT_EXCEEDED: code = Protocol.ReasonCode.BAD_PROTOCOL; break; case SYNC_FAILED: diff --git a/framework/src/main/java/org/tron/core/net/P2pRateLimiter.java b/framework/src/main/java/org/tron/core/net/P2pRateLimiter.java new file mode 100644 index 00000000000..9b36e1e5df3 --- /dev/null +++ b/framework/src/main/java/org/tron/core/net/P2pRateLimiter.java @@ -0,0 +1,32 @@ +package org.tron.core.net; + +import com.google.common.cache.Cache; +import com.google.common.cache.CacheBuilder; +import com.google.common.util.concurrent.RateLimiter; + +public class P2pRateLimiter { + private final Cache rateLimiters = CacheBuilder.newBuilder() + .maximumSize(32).build(); + + public void register(Byte type, double rate) { + RateLimiter rateLimiter = RateLimiter.create(Double.POSITIVE_INFINITY); + rateLimiter.setRate(rate); + rateLimiters.put(type, rateLimiter); + } + + public void acquire(Byte type) { + RateLimiter rateLimiter = rateLimiters.getIfPresent(type); + if (rateLimiter == null) { + return; + } + rateLimiter.acquire(); + } + + public boolean tryAcquire(Byte type) { + RateLimiter rateLimiter = rateLimiters.getIfPresent(type); + if (rateLimiter == null) { + return true; + } + return rateLimiter.tryAcquire(); + } +} diff --git a/framework/src/main/java/org/tron/core/net/message/handshake/HelloMessage.java b/framework/src/main/java/org/tron/core/net/message/handshake/HelloMessage.java index 867ced5dbff..68123c93db6 100755 --- a/framework/src/main/java/org/tron/core/net/message/handshake/HelloMessage.java +++ b/framework/src/main/java/org/tron/core/net/message/handshake/HelloMessage.java @@ -5,6 +5,7 @@ import org.apache.commons.lang3.StringUtils; import org.tron.common.utils.ByteArray; import org.tron.common.utils.DecodeUtil; +import org.tron.common.utils.Sha256Hash; import org.tron.common.utils.StringUtil; import org.tron.core.ChainBaseManager; import org.tron.core.capsule.BlockCapsule; @@ -156,17 +157,17 @@ public Protocol.HelloMessage getInstance() { public boolean valid() { byte[] genesisBlockByte = this.helloMessage.getGenesisBlockId().getHash().toByteArray(); - if (genesisBlockByte.length == 0) { + if (genesisBlockByte.length != Sha256Hash.LENGTH) { return false; } byte[] solidBlockId = this.helloMessage.getSolidBlockId().getHash().toByteArray(); - if (solidBlockId.length == 0) { + if (solidBlockId.length != Sha256Hash.LENGTH) { return false; } byte[] headBlockId = this.helloMessage.getHeadBlockId().getHash().toByteArray(); - if (headBlockId.length == 0) { + if (headBlockId.length != Sha256Hash.LENGTH) { return false; } diff --git a/framework/src/main/java/org/tron/core/net/messagehandler/FetchInvDataMsgHandler.java b/framework/src/main/java/org/tron/core/net/messagehandler/FetchInvDataMsgHandler.java index 5415ea435e3..ecb7853ce6f 100644 --- a/framework/src/main/java/org/tron/core/net/messagehandler/FetchInvDataMsgHandler.java +++ b/framework/src/main/java/org/tron/core/net/messagehandler/FetchInvDataMsgHandler.java @@ -38,7 +38,7 @@ public class FetchInvDataMsgHandler implements TronMsgHandler { private volatile Cache epochCache = CacheBuilder.newBuilder().initialCapacity(100) - .maximumSize(1000).expireAfterWrite(1, TimeUnit.HOURS).build(); + .maximumSize(1000).expireAfterWrite(1, TimeUnit.HOURS).build(); private static final int MAX_SIZE = 1_000_000; @Autowired @@ -55,7 +55,9 @@ public void processMessage(PeerConnection peer, TronMessage msg) throws P2pExcep FetchInvDataMessage fetchInvDataMsg = (FetchInvDataMessage) msg; - check(peer, fetchInvDataMsg); + boolean isAdv = isAdvInv(peer, fetchInvDataMsg); + + check(peer, fetchInvDataMsg, isAdv); InventoryType type = fetchInvDataMsg.getInventoryType(); List transactions = Lists.newArrayList(); @@ -64,6 +66,15 @@ public void processMessage(PeerConnection peer, TronMessage msg) throws P2pExcep for (Sha256Hash hash : fetchInvDataMsg.getHashList()) { Item item = new Item(hash, type); + /* Cache the Inventory sent to the peer. + Once a FetchInvData message is received from the peer, remove this Inventory from the cache. + If the same FetchInvData request is received from the peer again and it is + no longer in the cache, then reject the request. + * */ + if (isAdv) { + peer.getAdvInvSpread().invalidate(item); + } + Message message = advService.getMessage(item); if (message == null) { try { @@ -84,7 +95,7 @@ public void processMessage(PeerConnection peer, TronMessage msg) throws P2pExcep } else { transactions.add(((TransactionMessage) message).getTransactionCapsule().getInstance()); size += ((TransactionMessage) message).getTransactionCapsule().getInstance() - .getSerializedSize(); + .getSerializedSize(); if (size > MAX_SIZE) { peer.sendMessage(new TransactionsMessage(transactions)); transactions = Lists.newArrayList(); @@ -104,16 +115,16 @@ private void sendPbftCommitMessage(PeerConnection peer, BlockCapsule blockCapsul } long epoch = 0; PbftSignCapsule pbftSignCapsule = tronNetDelegate - .getBlockPbftCommitData(blockCapsule.getNum()); + .getBlockPbftCommitData(blockCapsule.getNum()); long maintenanceTimeInterval = consensusDelegate.getDynamicPropertiesStore() - .getMaintenanceTimeInterval(); + .getMaintenanceTimeInterval(); if (pbftSignCapsule != null) { Raw raw = Raw.parseFrom(pbftSignCapsule.getPbftCommitResult().getData()); epoch = raw.getEpoch(); peer.sendMessage(new PbftCommitMessage(pbftSignCapsule)); } else { - epoch = - (blockCapsule.getTimeStamp() / maintenanceTimeInterval + 1) * maintenanceTimeInterval; + epoch = (blockCapsule.getTimeStamp() / maintenanceTimeInterval + 1) + * maintenanceTimeInterval; } if (epochCache.getIfPresent(epoch) == null) { PbftSignCapsule srl = tronNetDelegate.getSRLPbftCommitData(epoch); @@ -127,7 +138,21 @@ private void sendPbftCommitMessage(PeerConnection peer, BlockCapsule blockCapsul } } - private void check(PeerConnection peer, FetchInvDataMessage fetchInvDataMsg) throws P2pException { + public boolean isAdvInv(PeerConnection peer, FetchInvDataMessage msg) { + MessageTypes type = msg.getInvMessageType(); + if (type == MessageTypes.TRX) { + return true; + } + for (Sha256Hash hash : msg.getHashList()) { + if (peer.getAdvInvSpread().getIfPresent(new Item(hash, InventoryType.BLOCK)) == null) { + return false; + } + } + return true; + } + + private void check(PeerConnection peer, FetchInvDataMessage fetchInvDataMsg, + boolean isAdv) throws P2pException { MessageTypes type = fetchInvDataMsg.getInvMessageType(); if (type == MessageTypes.TRX) { @@ -144,38 +169,38 @@ private void check(PeerConnection peer, FetchInvDataMessage fetchInvDataMsg) thr + "maxCount: {}, fetchCount: {}, peer: {}", maxCount, fetchCount, peer.getInetAddress()); } - } else { - boolean isAdv = true; + } + + if (!isAdv) { + if (!peer.isNeedSyncFromUs()) { + throw new P2pException(TypeEnum.BAD_MESSAGE, "no need sync"); + } + if (!peer.getP2pRateLimiter().tryAcquire(fetchInvDataMsg.getType().asByte())) { + throw new P2pException(TypeEnum.RATE_LIMIT_EXCEEDED, fetchInvDataMsg.getType() + + " message exceeds the rate limit"); + } + if (fetchInvDataMsg.getHashList().size() > NetConstants.MAX_BLOCK_FETCH_PER_PEER) { + throw new P2pException(TypeEnum.BAD_MESSAGE, "fetch too many blocks, size:" + + fetchInvDataMsg.getHashList().size()); + } for (Sha256Hash hash : fetchInvDataMsg.getHashList()) { - if (peer.getAdvInvSpread().getIfPresent(new Item(hash, InventoryType.BLOCK)) == null) { - isAdv = false; - break; + long blockNum = new BlockId(hash).getNum(); + long minBlockNum = + peer.getLastSyncBlockId().getNum() - 2 * NetConstants.SYNC_FETCH_BATCH_NUM; + if (blockNum < minBlockNum) { + throw new P2pException(TypeEnum.BAD_MESSAGE, + "minBlockNum: " + minBlockNum + ", blockNum: " + blockNum); } - } - if (!isAdv) { - if (!peer.isNeedSyncFromUs()) { - throw new P2pException(TypeEnum.BAD_MESSAGE, "no need sync"); + if (blockNum > peer.getLastSyncBlockId().getNum()) { + throw new P2pException(TypeEnum.BAD_MESSAGE, + "maxBlockNum: " + peer.getLastSyncBlockId().getNum() + ", blockNum: " + blockNum); } - for (Sha256Hash hash : fetchInvDataMsg.getHashList()) { - long blockNum = new BlockId(hash).getNum(); - long minBlockNum = - peer.getLastSyncBlockId().getNum() - 2 * NetConstants.SYNC_FETCH_BATCH_NUM; - if (blockNum < minBlockNum) { - throw new P2pException(TypeEnum.BAD_MESSAGE, - "minBlockNum: " + minBlockNum + ", blockNum: " + blockNum); - } - if (blockNum > peer.getLastSyncBlockId().getNum()) { - throw new P2pException(TypeEnum.BAD_MESSAGE, - "maxBlockNum: " + peer.getLastSyncBlockId().getNum() + ", blockNum: " + blockNum); - } - if (peer.getSyncBlockIdCache().getIfPresent(hash) != null) { - throw new P2pException(TypeEnum.BAD_MESSAGE, - new BlockId(hash).getString() + " is exist"); - } - peer.getSyncBlockIdCache().put(hash, System.currentTimeMillis()); + if (peer.getSyncBlockIdCache().getIfPresent(hash) != null) { + throw new P2pException(TypeEnum.BAD_MESSAGE, + new BlockId(hash).getString() + " is exist"); } + peer.getSyncBlockIdCache().put(hash, System.currentTimeMillis()); } } } - -} +} \ No newline at end of file diff --git a/framework/src/main/java/org/tron/core/net/messagehandler/SyncBlockChainMsgHandler.java b/framework/src/main/java/org/tron/core/net/messagehandler/SyncBlockChainMsgHandler.java index 55446593bd0..71d268b22bc 100644 --- a/framework/src/main/java/org/tron/core/net/messagehandler/SyncBlockChainMsgHandler.java +++ b/framework/src/main/java/org/tron/core/net/messagehandler/SyncBlockChainMsgHandler.java @@ -58,6 +58,14 @@ public void processMessage(PeerConnection peer, TronMessage msg) throws P2pExcep } private boolean check(PeerConnection peer, SyncBlockChainMessage msg) throws P2pException { + if (peer.getRemainNum() > 0 + && !peer.getP2pRateLimiter().tryAcquire(msg.getType().asByte())) { + // Discard messages that exceed the rate limit + logger.warn("{} message from peer {} exceeds the rate limit", + msg.getType(), peer.getInetSocketAddress()); + return false; + } + List blockIds = msg.getBlockIds(); if (CollectionUtils.isEmpty(blockIds)) { throw new P2pException(TypeEnum.BAD_MESSAGE, "SyncBlockChain blockIds is empty"); diff --git a/framework/src/main/java/org/tron/core/net/messagehandler/TransactionsMsgHandler.java b/framework/src/main/java/org/tron/core/net/messagehandler/TransactionsMsgHandler.java index 5fab8bc6f33..0436b48d374 100644 --- a/framework/src/main/java/org/tron/core/net/messagehandler/TransactionsMsgHandler.java +++ b/framework/src/main/java/org/tron/core/net/messagehandler/TransactionsMsgHandler.java @@ -70,6 +70,10 @@ public boolean isBusy() { public void processMessage(PeerConnection peer, TronMessage msg) throws P2pException { TransactionsMessage transactionsMessage = (TransactionsMessage) msg; check(peer, transactionsMessage); + for (Transaction trx : transactionsMessage.getTransactions().getTransactionsList()) { + Item item = new Item(new TransactionMessage(trx).getMessageId(), InventoryType.TRX); + peer.getAdvInvRequest().remove(item); + } int smartContractQueueSize = 0; int trxHandlePoolQueueSize = 0; int dropSmartContractCount = 0; @@ -101,7 +105,6 @@ private void check(PeerConnection peer, TransactionsMessage msg) throws P2pExcep throw new P2pException(TypeEnum.BAD_MESSAGE, "trx: " + msg.getMessageId() + " without request."); } - peer.getAdvInvRequest().remove(item); if (trx.getRawData().getContractCount() < 1) { throw new P2pException(TypeEnum.BAD_TRX, "tx " + item.getHash() + " contract size should be greater than 0"); diff --git a/framework/src/main/java/org/tron/core/net/peer/PeerConnection.java b/framework/src/main/java/org/tron/core/net/peer/PeerConnection.java index 2e08e105bed..253502bc3a1 100644 --- a/framework/src/main/java/org/tron/core/net/peer/PeerConnection.java +++ b/framework/src/main/java/org/tron/core/net/peer/PeerConnection.java @@ -1,5 +1,9 @@ package org.tron.core.net.peer; +import static org.tron.core.net.message.MessageTypes.FETCH_INV_DATA; +import static org.tron.core.net.message.MessageTypes.P2P_DISCONNECT; +import static org.tron.core.net.message.MessageTypes.SYNC_BLOCK_CHAIN; + import com.google.common.cache.Cache; import com.google.common.cache.CacheBuilder; import com.google.protobuf.ByteString; @@ -32,6 +36,7 @@ import org.tron.core.config.args.Args; import org.tron.core.metrics.MetricsKey; import org.tron.core.metrics.MetricsUtil; +import org.tron.core.net.P2pRateLimiter; import org.tron.core.net.TronNetDelegate; import org.tron.core.net.message.adv.InventoryMessage; import org.tron.core.net.message.adv.TransactionsMessage; @@ -85,7 +90,7 @@ public class PeerConnection { @Getter @Setter - private TronState tronState = TronState.INIT; + private volatile TronState tronState = TronState.INIT; @Autowired private TronNetDelegate tronNetDelegate; @@ -123,15 +128,15 @@ public class PeerConnection { private Map advInvRequest = new ConcurrentHashMap<>(); @Setter - private BlockId fastForwardBlock; + private volatile BlockId fastForwardBlock; @Getter - private BlockId blockBothHave = new BlockId(); + private volatile BlockId blockBothHave = new BlockId(); @Getter private volatile long blockBothHaveUpdateTime = System.currentTimeMillis(); @Setter @Getter - private BlockId lastSyncBlockId; + private volatile BlockId lastSyncBlockId; @Setter @Getter private volatile long remainNum; @@ -146,7 +151,7 @@ public class PeerConnection { private Map syncBlockRequested = new ConcurrentHashMap<>(); @Setter @Getter - private Pair, Long> syncChainRequested = null; + private volatile Pair, Long> syncChainRequested = null; @Setter @Getter private Set syncBlockInProcess = new HashSet<>(); @@ -156,6 +161,8 @@ public class PeerConnection { @Setter @Getter private volatile boolean needSyncFromUs = true; + @Getter + private P2pRateLimiter p2pRateLimiter = new P2pRateLimiter(); public void setChannel(Channel channel) { this.channel = channel; @@ -164,6 +171,12 @@ public void setChannel(Channel channel) { } this.nodeStatistics = TronStatsManager.getNodeStatistics(channel.getInetAddress()); lastInteractiveTime = System.currentTimeMillis(); + p2pRateLimiter.register(SYNC_BLOCK_CHAIN.asByte(), + Args.getInstance().getRateLimiterSyncBlockChain()); + p2pRateLimiter.register(FETCH_INV_DATA.asByte(), + Args.getInstance().getRateLimiterFetchInvData()); + p2pRateLimiter.register(P2P_DISCONNECT.asByte(), + Args.getInstance().getRateLimiterDisconnect()); } public void setBlockBothHave(BlockId blockId) { diff --git a/framework/src/main/java/org/tron/core/net/peer/PeerStatusCheck.java b/framework/src/main/java/org/tron/core/net/peer/PeerStatusCheck.java index 6ccbf6427a7..04eac202484 100644 --- a/framework/src/main/java/org/tron/core/net/peer/PeerStatusCheck.java +++ b/framework/src/main/java/org/tron/core/net/peer/PeerStatusCheck.java @@ -42,6 +42,10 @@ public void statusCheck() { long now = System.currentTimeMillis(); + if (tronNetDelegate == null) { + // only occurs in mock test. TODO fix test + return; + } tronNetDelegate.getActivePeer().forEach(peer -> { boolean isDisconnected = false; diff --git a/framework/src/main/java/org/tron/core/net/service/handshake/HandshakeService.java b/framework/src/main/java/org/tron/core/net/service/handshake/HandshakeService.java index 6cd117c83dd..070a9f56406 100644 --- a/framework/src/main/java/org/tron/core/net/service/handshake/HandshakeService.java +++ b/framework/src/main/java/org/tron/core/net/service/handshake/HandshakeService.java @@ -6,6 +6,7 @@ import org.springframework.stereotype.Component; import org.tron.common.utils.ByteArray; import org.tron.core.ChainBaseManager; +import org.tron.core.ChainBaseManager.NodeType; import org.tron.core.config.args.Args; import org.tron.core.net.TronNetService; import org.tron.core.net.message.handshake.HelloMessage; @@ -57,7 +58,7 @@ public void processHelloMessage(PeerConnection peer, HelloMessage msg) { msg.getInstance().getAddress().toByteArray().length, msg.getInstance().getSignature().toByteArray().length, msg.getInstance().getCodeVersion().toByteArray().length); - peer.disconnect(ReasonCode.UNEXPECTED_IDENTITY); + peer.disconnect(ReasonCode.INCOMPATIBLE_PROTOCOL); return; } @@ -96,12 +97,17 @@ public void processHelloMessage(PeerConnection peer, HelloMessage msg) { } if (chainBaseManager.getSolidBlockId().getNum() >= msg.getSolidBlockId().getNum() - && !chainBaseManager.containBlockInMainChain(msg.getSolidBlockId())) { - logger.info("Peer {} different solid block, peer->{}, me->{}", - peer.getInetSocketAddress(), - msg.getSolidBlockId().getString(), - chainBaseManager.getSolidBlockId().getString()); - peer.disconnect(ReasonCode.FORKED); + && !chainBaseManager.containBlockInMainChain(msg.getSolidBlockId())) { + if (chainBaseManager.getLowestBlockNum() <= msg.getSolidBlockId().getNum()) { + logger.info("Peer {} different solid block, fork with me, peer->{}, me->{}", + peer.getInetSocketAddress(), + msg.getSolidBlockId().getString(), + chainBaseManager.getSolidBlockId().getString()); + peer.disconnect(ReasonCode.FORKED); + } else { + logger.info("Peer {} solid block is below than my lowest", peer.getInetSocketAddress()); + peer.disconnect(ReasonCode.LIGHT_NODE_SYNC_FAIL); + } return; } diff --git a/framework/src/main/java/org/tron/core/net/service/relay/RelayService.java b/framework/src/main/java/org/tron/core/net/service/relay/RelayService.java index 161e918336b..61ae6326e9f 100644 --- a/framework/src/main/java/org/tron/core/net/service/relay/RelayService.java +++ b/framework/src/main/java/org/tron/core/net/service/relay/RelayService.java @@ -66,11 +66,11 @@ public class RelayService { private List fastForwardNodes = parameter.getFastForwardNodes(); - private ByteString witnessAddress = ByteString - .copyFrom(Args.getLocalWitnesses().getWitnessAccountAddress(CommonParameter.getInstance() - .isECKeyCryptoEngine())); + private final int keySize = Args.getLocalWitnesses().getPrivateKeys().size(); - private int keySize = Args.getLocalWitnesses().getPrivateKeys().size(); + private final ByteString witnessAddress = + Args.getLocalWitnesses().getWitnessAccountAddress() != null ? ByteString + .copyFrom(Args.getLocalWitnesses().getWitnessAccountAddress()) : null; private int maxFastForwardNum = Args.getInstance().getMaxFastForwardNum(); diff --git a/framework/src/main/java/org/tron/core/net/service/sync/SyncService.java b/framework/src/main/java/org/tron/core/net/service/sync/SyncService.java index e387329c467..75349bd4c19 100644 --- a/framework/src/main/java/org/tron/core/net/service/sync/SyncService.java +++ b/framework/src/main/java/org/tron/core/net/service/sync/SyncService.java @@ -124,7 +124,11 @@ public void syncNext(PeerConnection peer) { peer.setSyncChainRequested(new Pair<>(chainSummary, System.currentTimeMillis())); peer.sendMessage(new SyncBlockChainMessage(chainSummary)); } catch (Exception e) { - logger.error("Peer {} sync failed, reason: {}", peer.getInetAddress(), e); + if (e instanceof P2pException) { + logger.warn("Peer {} sync failed, reason: {}", peer.getInetAddress(), e.getMessage()); + } else { + logger.error("Peer {} sync failed.", peer.getInetAddress(), e); + } peer.disconnect(ReasonCode.SYNC_FAIL); } } @@ -159,9 +163,8 @@ private void invalid(BlockId blockId, PeerConnection peerConnection) { } private LinkedList getBlockChainSummary(PeerConnection peer) throws P2pException { - - BlockId beginBlockId = peer.getBlockBothHave(); List blockIds = new ArrayList<>(peer.getSyncBlockToFetch()); + BlockId beginBlockId = peer.getBlockBothHave(); List forkList = new LinkedList<>(); LinkedList summary = new LinkedList<>(); long syncBeginNumber = tronNetDelegate.getSyncBeginNumber(); @@ -323,9 +326,9 @@ private void processSyncBlock(BlockCapsule block, PeerConnection peerConnection) for (PeerConnection peer : tronNetDelegate.getActivePeer()) { BlockId bid = peer.getSyncBlockToFetch().peek(); if (blockId.equals(bid)) { + peer.setBlockBothHave(blockId); peer.getSyncBlockToFetch().remove(bid); if (flag) { - peer.setBlockBothHave(blockId); if (peer.getSyncBlockToFetch().isEmpty() && peer.isFetchAble()) { syncNext(peer); } diff --git a/framework/src/main/java/org/tron/core/services/RpcApiService.java b/framework/src/main/java/org/tron/core/services/RpcApiService.java index 8f9c6b15bb7..63e7ba03fc7 100755 --- a/framework/src/main/java/org/tron/core/services/RpcApiService.java +++ b/framework/src/main/java/org/tron/core/services/RpcApiService.java @@ -89,6 +89,7 @@ import org.tron.core.exception.ContractExeException; import org.tron.core.exception.ContractValidateException; import org.tron.core.exception.ItemNotFoundException; +import org.tron.core.exception.MaintenanceUnavailableException; import org.tron.core.exception.NonUniqueObjectException; import org.tron.core.exception.StoreException; import org.tron.core.exception.VMIllegalException; @@ -291,20 +292,6 @@ private StatusRuntimeException getRunTimeException(Exception e) { } } - private void checkSupportShieldedTransaction() throws ZksnarkException { - String msg = "Not support Shielded Transaction, need to be opened by the committee"; - if (!dbManager.getDynamicPropertiesStore().supportShieldedTransaction()) { - throw new ZksnarkException(msg); - } - } - - private void checkSupportShieldedTRC20Transaction() throws ZksnarkException { - String msg = "Not support Shielded TRC20 Transaction, need to be opened by the committee"; - if (!dbManager.getDynamicPropertiesStore().supportShieldedTRC20Transaction()) { - throw new ZksnarkException(msg); - } - } - /** * DatabaseApi. */ @@ -396,6 +383,18 @@ public void listWitnesses(EmptyMessage request, StreamObserver resp responseObserver.onCompleted(); } + @Override + public void getPaginatedNowWitnessList(PaginatedMessage request, + StreamObserver responseObserver) { + try { + responseObserver.onNext( + wallet.getPaginatedNowWitnessList(request.getOffset(), request.getLimit())); + } catch (MaintenanceUnavailableException e) { + responseObserver.onError(getRunTimeException(e)); + } + responseObserver.onCompleted(); + } + @Override public void getAssetIssueList(EmptyMessage request, StreamObserver responseObserver) { @@ -651,8 +650,6 @@ public void getMerkleTreeVoucherInfo(OutputPointInfo request, StreamObserver responseObserver) { try { - checkSupportShieldedTransaction(); - IncrementalMerkleVoucherInfo witnessInfo = wallet .getMerkleTreeVoucherInfo(request); responseObserver.onNext(witnessInfo); @@ -669,8 +666,6 @@ public void scanNoteByIvk(GrpcAPI.IvkDecryptParameters request, long endNum = request.getEndBlockIndex(); try { - checkSupportShieldedTransaction(); - DecryptNotes decryptNotes = wallet .scanNoteByIvk(startNum, endNum, request.getIvk().toByteArray()); responseObserver.onNext(decryptNotes); @@ -687,8 +682,6 @@ public void scanAndMarkNoteByIvk(GrpcAPI.IvkDecryptAndMarkParameters request, long endNum = request.getEndBlockIndex(); try { - checkSupportShieldedTransaction(); - DecryptNotesMarked decryptNotes = wallet.scanAndMarkNoteByIvk(startNum, endNum, request.getIvk().toByteArray(), request.getAk().toByteArray(), @@ -707,8 +700,6 @@ public void scanNoteByOvk(GrpcAPI.OvkDecryptParameters request, long startNum = request.getStartBlockIndex(); long endNum = request.getEndBlockIndex(); try { - checkSupportShieldedTransaction(); - DecryptNotes decryptNotes = wallet .scanNoteByOvk(startNum, endNum, request.getOvk().toByteArray()); responseObserver.onNext(decryptNotes); @@ -721,8 +712,6 @@ public void scanNoteByOvk(GrpcAPI.OvkDecryptParameters request, @Override public void isSpend(NoteParameters request, StreamObserver responseObserver) { try { - checkSupportShieldedTransaction(); - responseObserver.onNext(wallet.isSpend(request)); } catch (Exception e) { responseObserver.onError(getRunTimeException(e)); @@ -742,7 +731,6 @@ public void scanShieldedTRC20NotesByIvk(IvkDecryptTRC20Parameters request, ProtocolStringList topicsList = request.getEventsList(); try { - checkSupportShieldedTRC20Transaction(); responseObserver.onNext( wallet.scanShieldedTRC20NotesByIvk(startNum, endNum, contractAddress, ivk, ak, nk, topicsList)); @@ -762,7 +750,6 @@ public void scanShieldedTRC20NotesByOvk(OvkDecryptTRC20Parameters request, byte[] ovk = request.getOvk().toByteArray(); ProtocolStringList topicList = request.getEventsList(); try { - checkSupportShieldedTRC20Transaction(); responseObserver .onNext(wallet .scanShieldedTRC20NotesByOvk(startNum, endNum, ovk, contractAddress, topicList)); @@ -776,7 +763,6 @@ public void scanShieldedTRC20NotesByOvk(OvkDecryptTRC20Parameters request, public void isShieldedTRC20ContractNoteSpent(NfTRC20Parameters request, StreamObserver responseObserver) { try { - checkSupportShieldedTRC20Transaction(); responseObserver.onNext(wallet.isShieldedTRC20ContractNoteSpent(request)); } catch (Exception e) { responseObserver.onError(getRunTimeException(e)); @@ -1872,6 +1858,18 @@ public void listWitnesses(EmptyMessage request, responseObserver.onCompleted(); } + @Override + public void getPaginatedNowWitnessList(PaginatedMessage request, + StreamObserver responseObserver) { + try { + responseObserver.onNext( + wallet.getPaginatedNowWitnessList(request.getOffset(), request.getLimit())); + } catch (MaintenanceUnavailableException e) { + responseObserver.onError(getRunTimeException(e)); + } + responseObserver.onCompleted(); + } + @Override public void listProposals(EmptyMessage request, StreamObserver responseObserver) { @@ -2066,8 +2064,6 @@ public void getMerkleTreeVoucherInfo(OutputPointInfo request, StreamObserver responseObserver) { try { - checkSupportShieldedTransaction(); - IncrementalMerkleVoucherInfo witnessInfo = wallet .getMerkleTreeVoucherInfo(request); responseObserver.onNext(witnessInfo); @@ -2087,8 +2083,6 @@ public void createShieldedTransaction(PrivateParameters request, Return.Builder retBuilder = Return.newBuilder(); try { - checkSupportShieldedTransaction(); - TransactionCapsule trx = wallet.createShieldedTransaction(request); trxExtBuilder.setTransaction(trx.getInstance()); trxExtBuilder.setTxid(trx.getTransactionId().getByteString()); @@ -2118,8 +2112,6 @@ public void createShieldedTransactionWithoutSpendAuthSig(PrivateParametersWithou Return.Builder retBuilder = Return.newBuilder(); try { - checkSupportShieldedTransaction(); - TransactionCapsule trx = wallet.createShieldedTransactionWithoutSpendAuthSig(request); trxExtBuilder.setTransaction(trx.getInstance()); trxExtBuilder.setTxid(trx.getTransactionId().getByteString()); @@ -2147,8 +2139,6 @@ public void getNewShieldedAddress(EmptyMessage request, StreamObserver responseObserver) { try { - checkSupportShieldedTRC20Transaction(); - responseObserver.onNext(wallet.getNewShieldedAddress()); } catch (Exception e) { responseObserver.onError(getRunTimeException(e)); @@ -2161,8 +2151,6 @@ public void getNewShieldedAddress(EmptyMessage request, public void getSpendingKey(EmptyMessage request, StreamObserver responseObserver) { try { - checkSupportShieldedTRC20Transaction(); - responseObserver.onNext(wallet.getSpendingKey()); } catch (Exception e) { responseObserver.onError(getRunTimeException(e)); @@ -2175,8 +2163,6 @@ public void getSpendingKey(EmptyMessage request, public void getRcm(EmptyMessage request, StreamObserver responseObserver) { try { - checkSupportShieldedTRC20Transaction(); - responseObserver.onNext(wallet.getRcm()); } catch (Exception e) { responseObserver.onError(getRunTimeException(e)); @@ -2191,8 +2177,6 @@ public void getExpandedSpendingKey(BytesMessage request, ByteString spendingKey = request.getValue(); try { - checkSupportShieldedTRC20Transaction(); - ExpandedSpendingKeyMessage response = wallet.getExpandedSpendingKey(spendingKey); responseObserver.onNext(response); } catch (BadItemException | ZksnarkException e) { @@ -2208,8 +2192,6 @@ public void getAkFromAsk(BytesMessage request, StreamObserver resp ByteString ak = request.getValue(); try { - checkSupportShieldedTRC20Transaction(); - responseObserver.onNext(wallet.getAkFromAsk(ak)); } catch (BadItemException | ZksnarkException e) { responseObserver.onError(getRunTimeException(e)); @@ -2224,8 +2206,6 @@ public void getNkFromNsk(BytesMessage request, StreamObserver resp ByteString nk = request.getValue(); try { - checkSupportShieldedTRC20Transaction(); - responseObserver.onNext(wallet.getNkFromNsk(nk)); } catch (BadItemException | ZksnarkException e) { responseObserver.onError(getRunTimeException(e)); @@ -2242,8 +2222,6 @@ public void getIncomingViewingKey(ViewingKeyMessage request, ByteString nk = request.getNk(); try { - checkSupportShieldedTRC20Transaction(); - responseObserver.onNext(wallet.getIncomingViewingKey(ak.toByteArray(), nk.toByteArray())); } catch (ZksnarkException e) { responseObserver.onError(getRunTimeException(e)); @@ -2257,8 +2235,6 @@ public void getIncomingViewingKey(ViewingKeyMessage request, public void getDiversifier(EmptyMessage request, StreamObserver responseObserver) { try { - checkSupportShieldedTRC20Transaction(); - DiversifierMessage d = wallet.getDiversifier(); responseObserver.onNext(d); } catch (ZksnarkException e) { @@ -2276,8 +2252,6 @@ public void getZenPaymentAddress(IncomingViewingKeyDiversifierMessage request, DiversifierMessage d = request.getD(); try { - checkSupportShieldedTRC20Transaction(); - PaymentAddressMessage saplingPaymentAddressMessage = wallet.getPaymentAddress(new IncomingViewingKey(ivk.getIvk().toByteArray()), new DiversifierT(d.getD().toByteArray())); @@ -2298,8 +2272,6 @@ public void scanNoteByIvk(GrpcAPI.IvkDecryptParameters request, long endNum = request.getEndBlockIndex(); try { - checkSupportShieldedTransaction(); - DecryptNotes decryptNotes = wallet .scanNoteByIvk(startNum, endNum, request.getIvk().toByteArray()); responseObserver.onNext(decryptNotes); @@ -2318,8 +2290,6 @@ public void scanAndMarkNoteByIvk(GrpcAPI.IvkDecryptAndMarkParameters request, long endNum = request.getEndBlockIndex(); try { - checkSupportShieldedTransaction(); - DecryptNotesMarked decryptNotes = wallet.scanAndMarkNoteByIvk(startNum, endNum, request.getIvk().toByteArray(), request.getAk().toByteArray(), @@ -2340,8 +2310,6 @@ public void scanNoteByOvk(GrpcAPI.OvkDecryptParameters request, long endNum = request.getEndBlockIndex(); try { - checkSupportShieldedTransaction(); - DecryptNotes decryptNotes = wallet .scanNoteByOvk(startNum, endNum, request.getOvk().toByteArray()); responseObserver.onNext(decryptNotes); @@ -2355,8 +2323,6 @@ public void scanNoteByOvk(GrpcAPI.OvkDecryptParameters request, @Override public void isSpend(NoteParameters request, StreamObserver responseObserver) { try { - checkSupportShieldedTransaction(); - responseObserver.onNext(wallet.isSpend(request)); } catch (Exception e) { responseObserver.onError(getRunTimeException(e)); @@ -2369,8 +2335,6 @@ public void isSpend(NoteParameters request, StreamObserver response public void createShieldNullifier(GrpcAPI.NfParameters request, StreamObserver responseObserver) { try { - checkSupportShieldedTransaction(); - BytesMessage nf = wallet .createShieldNullifier(request); responseObserver.onNext(nf); @@ -2385,8 +2349,6 @@ public void createShieldNullifier(GrpcAPI.NfParameters request, public void createSpendAuthSig(SpendAuthSigParameters request, StreamObserver responseObserver) { try { - checkSupportShieldedTRC20Transaction(); - BytesMessage spendAuthSig = wallet.createSpendAuthSig(request); responseObserver.onNext(spendAuthSig); } catch (Exception e) { @@ -2400,8 +2362,6 @@ public void createSpendAuthSig(SpendAuthSigParameters request, public void getShieldTransactionHash(Transaction request, StreamObserver responseObserver) { try { - checkSupportShieldedTransaction(); - BytesMessage transactionHash = wallet.getShieldTransactionHash(request); responseObserver.onNext(transactionHash); } catch (Exception e) { @@ -2416,8 +2376,6 @@ public void createShieldedContractParameters( PrivateShieldedTRC20Parameters request, StreamObserver responseObserver) { try { - checkSupportShieldedTRC20Transaction(); - ShieldedTRC20Parameters shieldedTRC20Parameters = wallet .createShieldedContractParameters(request); responseObserver.onNext(shieldedTRC20Parameters); @@ -2438,8 +2396,6 @@ public void createShieldedContractParametersWithoutAsk( PrivateShieldedTRC20ParametersWithoutAsk request, StreamObserver responseObserver) { try { - checkSupportShieldedTRC20Transaction(); - ShieldedTRC20Parameters shieldedTRC20Parameters = wallet .createShieldedContractParametersWithoutAsk(request); responseObserver.onNext(shieldedTRC20Parameters); @@ -2459,8 +2415,6 @@ public void scanShieldedTRC20NotesByIvk( long startNum = request.getStartBlockIndex(); long endNum = request.getEndBlockIndex(); try { - checkSupportShieldedTRC20Transaction(); - DecryptNotesTRC20 decryptNotes = wallet.scanShieldedTRC20NotesByIvk(startNum, endNum, request.getShieldedTRC20ContractAddress().toByteArray(), request.getIvk().toByteArray(), @@ -2487,8 +2441,6 @@ public void scanShieldedTRC20NotesByOvk( long startNum = request.getStartBlockIndex(); long endNum = request.getEndBlockIndex(); try { - checkSupportShieldedTRC20Transaction(); - DecryptNotesTRC20 decryptNotes = wallet.scanShieldedTRC20NotesByOvk(startNum, endNum, request.getOvk().toByteArray(), request.getShieldedTRC20ContractAddress().toByteArray(), @@ -2506,8 +2458,6 @@ public void scanShieldedTRC20NotesByOvk( public void isShieldedTRC20ContractNoteSpent(NfTRC20Parameters request, StreamObserver responseObserver) { try { - checkSupportShieldedTRC20Transaction(); - GrpcAPI.NullifierResult nf = wallet .isShieldedTRC20ContractNoteSpent(request); responseObserver.onNext(nf); @@ -2523,8 +2473,6 @@ public void getTriggerInputForShieldedTRC20Contract( ShieldedTRC20TriggerContractParameters request, StreamObserver responseObserver) { try { - checkSupportShieldedTRC20Transaction(); - responseObserver.onNext(wallet.getTriggerInputForShieldedTRC20Contract(request)); } catch (Exception e) { responseObserver.onError(e); diff --git a/framework/src/main/java/org/tron/core/services/event/BlockEventCache.java b/framework/src/main/java/org/tron/core/services/event/BlockEventCache.java index 3548859262e..e92bf6c8f1a 100644 --- a/framework/src/main/java/org/tron/core/services/event/BlockEventCache.java +++ b/framework/src/main/java/org/tron/core/services/event/BlockEventCache.java @@ -66,7 +66,14 @@ public static void add(BlockEvent blockEvent) throws EventException { } if (blockEvent.getSolidId().getNum() > solidId.getNum()) { - solidId = blockEvent.getSolidId(); + BlockCapsule.BlockId headBlockId = head.getBlockId(); + if (blockEvent.getSolidId().getNum() <= headBlockId.getNum()) { + solidId = blockEvent.getSolidId(); + } else if (blockEvent.getBlockId().equals(headBlockId)) { + // Fork chains needs to be considered to ensure that the head is on the main chain. + logger.info("Set solidId to head {}", headBlockId.getString()); + solidId = headBlockId; + } } } diff --git a/framework/src/main/java/org/tron/core/services/event/BlockEventGet.java b/framework/src/main/java/org/tron/core/services/event/BlockEventGet.java index bf668a3e0b6..122a61222c3 100644 --- a/framework/src/main/java/org/tron/core/services/event/BlockEventGet.java +++ b/framework/src/main/java/org/tron/core/services/event/BlockEventGet.java @@ -57,10 +57,16 @@ public BlockEvent getBlockEvent(long blockNum) throws Exception { BlockCapsule block = manager.getChainBaseManager().getBlockByNum(blockNum); block.getTransactions().forEach(t -> t.setBlockNum(block.getNum())); long solidNum = manager.getDynamicPropertiesStore().getLatestSolidifiedBlockNum(); + long headNum = manager.getHeadBlockNum(); + // solve the single SR concurrency problem + if (solidNum >= headNum && headNum > 0) { + solidNum = headNum - 1; + } BlockEvent blockEvent = new BlockEvent(); blockEvent.setBlockId(block.getBlockId()); blockEvent.setParentId(block.getParentBlockId()); blockEvent.setSolidId(manager.getChainBaseManager().getBlockIdByNum(solidNum)); + if (instance.isBlockLogTriggerEnable()) { blockEvent.setBlockLogTriggerCapsule(getBlockLogTrigger(block, solidNum)); } @@ -88,18 +94,13 @@ public BlockEvent getBlockEvent(long blockNum) throws Exception { } public SmartContractTrigger getContractTrigger(BlockCapsule block, long solidNum) { - TransactionRetCapsule result; - try { - result = manager.getChainBaseManager().getTransactionRetStore() - .getTransactionInfoByBlockNum(ByteArray.fromLong(block.getNum())); - } catch (BadItemException e) { - throw new RuntimeException(e); - } + + GrpcAPI.TransactionInfoList list = manager.getTransactionInfoByBlockNum(block.getNum()); SmartContractTrigger contractTrigger = new SmartContractTrigger(); for (int i = 0; i < block.getTransactions().size(); i++) { Protocol.Transaction tx = block.getInstance().getTransactions(i); - Protocol.TransactionInfo txInfo = result.getInstance().getTransactioninfo(i); + Protocol.TransactionInfo txInfo = list.getTransactionInfo(i); List triggers = parseLogs(tx, txInfo); for (ContractTrigger trigger : triggers) { @@ -328,22 +329,10 @@ public List getTransactionLogTrigger(BlockCapsule if (!EventPluginLoader.getInstance().isTransactionLogTriggerEthCompatible()) { return getTransactionTriggers(block, solidNum); } + + GrpcAPI.TransactionInfoList transactionInfoList + = manager.getTransactionInfoByBlockNum(block.getNum()); List transactionCapsuleList = block.getTransactions(); - GrpcAPI.TransactionInfoList transactionInfoList = GrpcAPI - .TransactionInfoList.newBuilder().build(); - GrpcAPI.TransactionInfoList.Builder transactionInfoListBuilder = GrpcAPI - .TransactionInfoList.newBuilder(); - try { - TransactionRetCapsule result = manager.getChainBaseManager().getTransactionRetStore() - .getTransactionInfoByBlockNum(ByteArray.fromLong(block.getNum())); - if (!Objects.isNull(result) && !Objects.isNull(result.getInstance())) { - result.getInstance().getTransactioninfoList() - .forEach(transactionInfoListBuilder::addTransactionInfo); - transactionInfoList = transactionInfoListBuilder.build(); - } - } catch (BadItemException e) { - logger.error("Get TransactionInfo failed, blockNum {}, {}.", block.getNum(), e.getMessage()); - } if (transactionCapsuleList.size() != transactionInfoList.getTransactionInfoCount()) { logger.error("Get TransactionInfo size not eq, blockNum {}, {}, {}", block.getNum(), transactionCapsuleList.size(), @@ -384,22 +373,8 @@ public List getTransactionTriggers(BlockCapsule bl return list; } - GrpcAPI.TransactionInfoList transactionInfoList = GrpcAPI - .TransactionInfoList.newBuilder().build(); - GrpcAPI.TransactionInfoList.Builder transactionInfoListBuilder = GrpcAPI - .TransactionInfoList.newBuilder(); - try { - TransactionRetCapsule result = manager.getChainBaseManager().getTransactionRetStore() - .getTransactionInfoByBlockNum(ByteArray.fromLong(block.getNum())); - if (!Objects.isNull(result) && !Objects.isNull(result.getInstance())) { - result.getInstance().getTransactioninfoList() - .forEach(transactionInfoListBuilder::addTransactionInfo); - transactionInfoList = transactionInfoListBuilder.build(); - } - } catch (Exception e) { - logger.warn("Get TransactionInfo failed, blockNum {}, {}.", block.getNum(), e.getMessage()); - } - + GrpcAPI.TransactionInfoList transactionInfoList + = manager.getTransactionInfoByBlockNum(block.getNum()); if (block.getTransactions().size() != transactionInfoList.getTransactionInfoCount()) { for (TransactionCapsule t : block.getTransactions()) { TransactionLogTriggerCapsule triggerCapsule = new TransactionLogTriggerCapsule(t, block); diff --git a/framework/src/main/java/org/tron/core/services/event/HistoryEventService.java b/framework/src/main/java/org/tron/core/services/event/HistoryEventService.java index 8f79ee47a3c..2eccb9fa2a9 100644 --- a/framework/src/main/java/org/tron/core/services/event/HistoryEventService.java +++ b/framework/src/main/java/org/tron/core/services/event/HistoryEventService.java @@ -29,6 +29,8 @@ public class HistoryEventService { @Autowired private Manager manager; + private volatile boolean isClosed = false; + private volatile Thread thread; public void init() { @@ -44,8 +46,15 @@ public void init() { } public void close() { + isClosed = true; if (thread != null) { - thread.interrupt(); + try { + thread.interrupt(); + thread.join(1000); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + logger.warn("Wait close timeout, {}", e.getMessage()); + } } logger.info("History event service close."); } @@ -54,7 +63,10 @@ private void syncEvent() { try { long tmp = instance.getStartSyncBlockNum(); long endNum = manager.getDynamicPropertiesStore().getLatestSolidifiedBlockNum(); - while (tmp <= endNum) { + while (tmp < endNum) { + if (thread.isInterrupted() || isClosed) { + throw new InterruptedException(); + } if (instance.isUseNativeQueue()) { Thread.sleep(20); } else if (instance.isBusy()) { @@ -67,7 +79,8 @@ private void syncEvent() { tmp++; endNum = manager.getDynamicPropertiesStore().getLatestSolidifiedBlockNum(); } - initEventService(manager.getChainBaseManager().getBlockIdByNum(endNum)); + long startNum = endNum == 0 ? 0 : endNum - 1; + initEventService(manager.getChainBaseManager().getBlockIdByNum(startNum)); } catch (InterruptedException e1) { logger.warn("History event service interrupted."); Thread.currentThread().interrupt(); diff --git a/framework/src/main/java/org/tron/core/services/event/RealtimeEventService.java b/framework/src/main/java/org/tron/core/services/event/RealtimeEventService.java index 093594f1c95..5aee55b1c13 100644 --- a/framework/src/main/java/org/tron/core/services/event/RealtimeEventService.java +++ b/framework/src/main/java/org/tron/core/services/event/RealtimeEventService.java @@ -4,6 +4,8 @@ import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.ScheduledExecutorService; import java.util.concurrent.TimeUnit; + +import lombok.Getter; import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; @@ -20,6 +22,9 @@ public class RealtimeEventService { private EventPluginLoader instance = EventPluginLoader.getInstance(); + @Getter + private static Object contractLock = new Object(); + @Autowired private Manager manager; @@ -94,30 +99,32 @@ public void flush(BlockEvent blockEvent, boolean isRemove) { } } - if (instance.isContractEventTriggerEnable()) { - if (blockEvent.getSmartContractTrigger() == null) { - logger.warn("SmartContractTrigger is null. {}", blockEvent.getBlockId().getString()); - } else { - blockEvent.getSmartContractTrigger().getContractEventTriggers().forEach(v -> { - v.setTriggerName(Trigger.CONTRACTEVENT_TRIGGER_NAME); - v.setRemoved(isRemove); - EventPluginLoader.getInstance().postContractEventTrigger(v); - }); + synchronized (contractLock) { + if (instance.isContractEventTriggerEnable()) { + if (blockEvent.getSmartContractTrigger() == null) { + logger.warn("SmartContractTrigger is null. {}", blockEvent.getBlockId().getString()); + } else { + blockEvent.getSmartContractTrigger().getContractEventTriggers().forEach(v -> { + v.setTriggerName(Trigger.CONTRACTEVENT_TRIGGER_NAME); + v.setRemoved(isRemove); + EventPluginLoader.getInstance().postContractEventTrigger(v); + }); + } } - } - if (instance.isContractLogTriggerEnable() && blockEvent.getSmartContractTrigger() != null) { - blockEvent.getSmartContractTrigger().getContractLogTriggers().forEach(v -> { - v.setTriggerName(Trigger.CONTRACTLOG_TRIGGER_NAME); - v.setRemoved(isRemove); - EventPluginLoader.getInstance().postContractLogTrigger(v); - }); - if (instance.isContractLogTriggerRedundancy()) { - blockEvent.getSmartContractTrigger().getRedundancies().forEach(v -> { + if (instance.isContractLogTriggerEnable() && blockEvent.getSmartContractTrigger() != null) { + blockEvent.getSmartContractTrigger().getContractLogTriggers().forEach(v -> { v.setTriggerName(Trigger.CONTRACTLOG_TRIGGER_NAME); v.setRemoved(isRemove); EventPluginLoader.getInstance().postContractLogTrigger(v); }); + if (instance.isContractLogTriggerRedundancy()) { + blockEvent.getSmartContractTrigger().getRedundancies().forEach(v -> { + v.setTriggerName(Trigger.CONTRACTLOG_TRIGGER_NAME); + v.setRemoved(isRemove); + EventPluginLoader.getInstance().postContractLogTrigger(v); + }); + } } } } diff --git a/framework/src/main/java/org/tron/core/services/event/SolidEventService.java b/framework/src/main/java/org/tron/core/services/event/SolidEventService.java index 6102a87f892..0614541f16f 100644 --- a/framework/src/main/java/org/tron/core/services/event/SolidEventService.java +++ b/framework/src/main/java/org/tron/core/services/event/SolidEventService.java @@ -86,27 +86,32 @@ public void flush(BlockEvent blockEvent) { } } - if (instance.isSolidityEventTriggerEnable()) { - if (blockEvent.getSmartContractTrigger() == null) { - logger.warn("SmartContractTrigger is null. {}", blockEvent.getBlockId()); - } else { - blockEvent.getSmartContractTrigger().getContractEventTriggers().forEach(v -> { - v.setTriggerName(Trigger.SOLIDITYEVENT_TRIGGER_NAME); - EventPluginLoader.getInstance().postSolidityEventTrigger(v); - }); + synchronized (RealtimeEventService.getContractLock()) { + if (instance.isSolidityEventTriggerEnable()) { + if (blockEvent.getSmartContractTrigger() == null) { + logger.warn("SmartContractTrigger is null. {}", blockEvent.getBlockId()); + } else { + blockEvent.getSmartContractTrigger().getContractEventTriggers().forEach(v -> { + v.setTriggerName(Trigger.SOLIDITYEVENT_TRIGGER_NAME); + v.setRemoved(false); + EventPluginLoader.getInstance().postSolidityEventTrigger(v); + }); + } } - } - if (instance.isSolidityLogTriggerEnable() && blockEvent.getSmartContractTrigger() != null) { - blockEvent.getSmartContractTrigger().getContractLogTriggers().forEach(v -> { - v.setTriggerName(Trigger.SOLIDITYLOG_TRIGGER_NAME); - EventPluginLoader.getInstance().postSolidityLogTrigger(v); - }); - if (instance.isSolidityLogTriggerRedundancy()) { - blockEvent.getSmartContractTrigger().getRedundancies().forEach(v -> { + if (instance.isSolidityLogTriggerEnable() && blockEvent.getSmartContractTrigger() != null) { + blockEvent.getSmartContractTrigger().getContractLogTriggers().forEach(v -> { v.setTriggerName(Trigger.SOLIDITYLOG_TRIGGER_NAME); + v.setRemoved(false); EventPluginLoader.getInstance().postSolidityLogTrigger(v); }); + if (instance.isSolidityLogTriggerRedundancy()) { + blockEvent.getSmartContractTrigger().getRedundancies().forEach(v -> { + v.setTriggerName(Trigger.SOLIDITYLOG_TRIGGER_NAME); + v.setRemoved(false); + EventPluginLoader.getInstance().postSolidityLogTrigger(v); + }); + } } } diff --git a/framework/src/main/java/org/tron/core/services/filter/HttpApiAccessFilter.java b/framework/src/main/java/org/tron/core/services/filter/HttpApiAccessFilter.java index 0405165ff99..59b9b15582b 100644 --- a/framework/src/main/java/org/tron/core/services/filter/HttpApiAccessFilter.java +++ b/framework/src/main/java/org/tron/core/services/filter/HttpApiAccessFilter.java @@ -26,7 +26,8 @@ public void init(FilterConfig filterConfig) { public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) { try { if (request instanceof HttpServletRequest) { - String endpoint = ((HttpServletRequest) request).getRequestURI(); + String contextPath = ((HttpServletRequest) request).getContextPath(); + String endpoint = contextPath + ((HttpServletRequest) request).getServletPath(); HttpServletResponse resp = (HttpServletResponse) response; if (isDisabled(endpoint)) { diff --git a/framework/src/main/java/org/tron/core/services/filter/HttpInterceptor.java b/framework/src/main/java/org/tron/core/services/filter/HttpInterceptor.java index 8b43cfef642..2ff8a5ad321 100644 --- a/framework/src/main/java/org/tron/core/services/filter/HttpInterceptor.java +++ b/framework/src/main/java/org/tron/core/services/filter/HttpInterceptor.java @@ -34,7 +34,8 @@ public void doFilter(ServletRequest request, ServletResponse response, FilterCha chain.doFilter(request, response); return; } - endpoint = ((HttpServletRequest) request).getRequestURI(); + String contextPath = ((HttpServletRequest) request).getContextPath(); + endpoint = contextPath + ((HttpServletRequest) request).getServletPath(); CharResponseWrapper responseWrapper = new CharResponseWrapper( (HttpServletResponse) response); chain.doFilter(request, responseWrapper); diff --git a/framework/src/main/java/org/tron/core/services/filter/LiteFnQueryHttpFilter.java b/framework/src/main/java/org/tron/core/services/filter/LiteFnQueryHttpFilter.java index a8ab947066c..07025996677 100644 --- a/framework/src/main/java/org/tron/core/services/filter/LiteFnQueryHttpFilter.java +++ b/framework/src/main/java/org/tron/core/services/filter/LiteFnQueryHttpFilter.java @@ -110,7 +110,8 @@ public void init(FilterConfig filterConfig) throws ServletException { @Override public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException { - String requestPath = ((HttpServletRequest) servletRequest).getRequestURI(); + String contextPath = ((HttpServletRequest) servletRequest).getContextPath(); + String requestPath = contextPath + ((HttpServletRequest) servletRequest).getServletPath(); if (chainBaseManager.isLiteNode() && !CommonParameter.getInstance().openHistoryQueryWhenLiteFN && filterPaths.contains(requestPath)) { diff --git a/framework/src/main/java/org/tron/core/services/http/FullNodeHttpApiService.java b/framework/src/main/java/org/tron/core/services/http/FullNodeHttpApiService.java index 76785218096..3ad4ace62fc 100644 --- a/framework/src/main/java/org/tron/core/services/http/FullNodeHttpApiService.java +++ b/framework/src/main/java/org/tron/core/services/http/FullNodeHttpApiService.java @@ -86,6 +86,8 @@ public class FullNodeHttpApiService extends HttpService { @Autowired private ListWitnessesServlet listWitnessesServlet; @Autowired + private GetPaginatedNowWitnessListServlet getPaginatedNowWitnessListServlet; + @Autowired private GetAssetIssueListServlet getAssetIssueListServlet; @Autowired private GetPaginatedAssetIssueListServlet getPaginatedAssetIssueListServlet; @@ -342,7 +344,11 @@ protected void addServlet(ServletContextHandler context) { context.addServlet( new ServletHolder(getTransactionCountByBlockNumServlet), "/wallet/gettransactioncountbyblocknum"); + // Get the list of witnesses info with contains vote counts for last epoch/maintenance context.addServlet(new ServletHolder(listWitnessesServlet), "/wallet/listwitnesses"); + // Get the paged list of witnesses info with realtime vote counts + context.addServlet(new ServletHolder(getPaginatedNowWitnessListServlet), + "/wallet/getpaginatednowwitnesslist"); context.addServlet(new ServletHolder(getAssetIssueListServlet), "/wallet/getassetissuelist"); context.addServlet( new ServletHolder(getPaginatedAssetIssueListServlet), diff --git a/framework/src/main/java/org/tron/core/services/http/GetPaginatedNowWitnessListServlet.java b/framework/src/main/java/org/tron/core/services/http/GetPaginatedNowWitnessListServlet.java new file mode 100644 index 00000000000..e53ab6610ec --- /dev/null +++ b/framework/src/main/java/org/tron/core/services/http/GetPaginatedNowWitnessListServlet.java @@ -0,0 +1,52 @@ +package org.tron.core.services.http; + +import java.io.IOException; +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.stereotype.Component; +import org.tron.api.GrpcAPI; +import org.tron.core.Wallet; +import org.tron.core.exception.MaintenanceUnavailableException; + +// Get the paged list of witnesses info with realtime vote counts +@Component +@Slf4j(topic = "API") +public class GetPaginatedNowWitnessListServlet extends RateLimiterServlet { + + @Autowired + private Wallet wallet; + + protected void doGet(HttpServletRequest request, HttpServletResponse response) { + try { + boolean visible = Util.getVisible(request); + long offset = Long.parseLong(request.getParameter("offset")); + long limit = Long.parseLong(request.getParameter("limit")); + fillResponse(offset, limit, visible, response); + } catch (Exception e) { + Util.processError(e, response); + } + } + + protected void doPost(HttpServletRequest request, HttpServletResponse response) { + try { + PostParams params = PostParams.getPostParams(request); + GrpcAPI.PaginatedMessage.Builder build = GrpcAPI.PaginatedMessage.newBuilder(); + JsonFormat.merge(params.getParams(), build, params.isVisible()); + fillResponse(build.getOffset(), build.getLimit(), params.isVisible(), response); + } catch (Exception e) { + Util.processError(e, response); + } + } + + private void fillResponse(long offset, long limit, boolean visible, HttpServletResponse response) + throws IOException, MaintenanceUnavailableException { + GrpcAPI.WitnessList reply = wallet.getPaginatedNowWitnessList(offset, limit); + if (reply != null) { + response.getWriter().println(JsonFormat.printToString(reply, visible)); + } else { + response.getWriter().println("{}"); + } + } +} diff --git a/framework/src/main/java/org/tron/core/services/http/GetProposalByIdServlet.java b/framework/src/main/java/org/tron/core/services/http/GetProposalByIdServlet.java index a903a5b4920..9d7805d4f98 100644 --- a/framework/src/main/java/org/tron/core/services/http/GetProposalByIdServlet.java +++ b/framework/src/main/java/org/tron/core/services/http/GetProposalByIdServlet.java @@ -26,7 +26,7 @@ protected void doGet(HttpServletRequest request, HttpServletResponse response) { try { boolean visible = Util.getVisible(request); String input = request.getParameter("id"); - long id = new Long(input); + long id = Long.parseLong(input); fillResponse(ByteString.copyFrom(ByteArray.fromLong(id)), visible, response); } catch (Exception e) { Util.processError(e, response); diff --git a/framework/src/main/java/org/tron/core/services/http/RateLimiterServlet.java b/framework/src/main/java/org/tron/core/services/http/RateLimiterServlet.java index fa59a72303d..7a66aed34f6 100644 --- a/framework/src/main/java/org/tron/core/services/http/RateLimiterServlet.java +++ b/framework/src/main/java/org/tron/core/services/http/RateLimiterServlet.java @@ -99,8 +99,9 @@ protected void service(HttpServletRequest req, HttpServletResponse resp) if (rateLimiter != null) { acquireResource = rateLimiter.acquire(runtimeData); } - String url = Strings.isNullOrEmpty(req.getRequestURI()) - ? MetricLabels.UNDEFINED : req.getRequestURI(); + String contextPath = req.getContextPath(); + String url = Strings.isNullOrEmpty(req.getServletPath()) + ? MetricLabels.UNDEFINED : contextPath + req.getServletPath(); try { resp.setContentType("application/json; charset=utf-8"); diff --git a/framework/src/main/java/org/tron/core/services/http/solidity/SolidityNodeHttpApiService.java b/framework/src/main/java/org/tron/core/services/http/solidity/SolidityNodeHttpApiService.java index ea08d2d42cf..359adfc2b39 100644 --- a/framework/src/main/java/org/tron/core/services/http/solidity/SolidityNodeHttpApiService.java +++ b/framework/src/main/java/org/tron/core/services/http/solidity/SolidityNodeHttpApiService.java @@ -44,6 +44,7 @@ import org.tron.core.services.http.GetNodeInfoServlet; import org.tron.core.services.http.GetNowBlockServlet; import org.tron.core.services.http.GetPaginatedAssetIssueListServlet; +import org.tron.core.services.http.GetPaginatedNowWitnessListServlet; import org.tron.core.services.http.GetRewardServlet; import org.tron.core.services.http.GetTransactionCountByBlockNumServlet; import org.tron.core.services.http.GetTransactionInfoByBlockNumServlet; @@ -92,6 +93,8 @@ public class SolidityNodeHttpApiService extends HttpService { @Autowired private ListWitnessesServlet listWitnessesServlet; @Autowired + private GetPaginatedNowWitnessListServlet getPaginatedNowWitnessListServlet; + @Autowired private GetAssetIssueListServlet getAssetIssueListServlet; @Autowired private GetPaginatedAssetIssueListServlet getPaginatedAssetIssueListServlet; @@ -174,6 +177,8 @@ protected void addServlet(ServletContextHandler context) { // same as FullNode context.addServlet(new ServletHolder(getAccountServlet), "/walletsolidity/getaccount"); context.addServlet(new ServletHolder(listWitnessesServlet), "/walletsolidity/listwitnesses"); + context.addServlet(new ServletHolder(getPaginatedNowWitnessListServlet), + "/walletsolidity/getpaginatednowwitnesslist"); context.addServlet(new ServletHolder(getAssetIssueListServlet), "/walletsolidity/getassetissuelist"); context.addServlet(new ServletHolder(getPaginatedAssetIssueListServlet), diff --git a/framework/src/main/java/org/tron/core/services/interfaceOnPBFT/http/PBFT/HttpApiOnPBFTService.java b/framework/src/main/java/org/tron/core/services/interfaceOnPBFT/http/PBFT/HttpApiOnPBFTService.java index 828d36e664f..a77b45353c9 100644 --- a/framework/src/main/java/org/tron/core/services/interfaceOnPBFT/http/PBFT/HttpApiOnPBFTService.java +++ b/framework/src/main/java/org/tron/core/services/interfaceOnPBFT/http/PBFT/HttpApiOnPBFTService.java @@ -172,7 +172,7 @@ public class HttpApiOnPBFTService extends HttpService { public HttpApiOnPBFTService() { port = Args.getInstance().getPBFTHttpPort(); enable = isFullNode() && Args.getInstance().isPBFTHttpEnable(); - contextPath = "/walletpbft/"; + contextPath = "/walletpbft"; } @Override diff --git a/framework/src/main/java/org/tron/core/services/interfaceOnSolidity/RpcApiServiceOnSolidity.java b/framework/src/main/java/org/tron/core/services/interfaceOnSolidity/RpcApiServiceOnSolidity.java index aa566f56042..315d70df8d6 100755 --- a/framework/src/main/java/org/tron/core/services/interfaceOnSolidity/RpcApiServiceOnSolidity.java +++ b/framework/src/main/java/org/tron/core/services/interfaceOnSolidity/RpcApiServiceOnSolidity.java @@ -162,6 +162,13 @@ public void listWitnesses(EmptyMessage request, StreamObserver resp () -> rpcApiService.getWalletSolidityApi().listWitnesses(request, responseObserver)); } + public void getPaginatedNowWitnessList(PaginatedMessage request, + StreamObserver responseObserver) { + walletOnSolidity.futureGet( + () -> rpcApiService.getWalletSolidityApi() + .getPaginatedNowWitnessList(request, responseObserver)); + } + @Override public void getAssetIssueById(BytesMessage request, StreamObserver responseObserver) { diff --git a/framework/src/main/java/org/tron/core/services/interfaceOnSolidity/http/GetPaginatedNowWitnessListOnSolidityServlet.java b/framework/src/main/java/org/tron/core/services/interfaceOnSolidity/http/GetPaginatedNowWitnessListOnSolidityServlet.java new file mode 100644 index 00000000000..4578393ec76 --- /dev/null +++ b/framework/src/main/java/org/tron/core/services/interfaceOnSolidity/http/GetPaginatedNowWitnessListOnSolidityServlet.java @@ -0,0 +1,24 @@ +package org.tron.core.services.interfaceOnSolidity.http; + +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.stereotype.Component; +import org.tron.core.services.http.GetPaginatedNowWitnessListServlet; +import org.tron.core.services.interfaceOnSolidity.WalletOnSolidity; + +@Component +@Slf4j(topic = "API") +public class GetPaginatedNowWitnessListOnSolidityServlet extends GetPaginatedNowWitnessListServlet { + @Autowired + private WalletOnSolidity walletOnSolidity; + + protected void doGet(HttpServletRequest request, HttpServletResponse response) { + walletOnSolidity.futureGet(() -> super.doGet(request, response)); + } + + protected void doPost(HttpServletRequest request, HttpServletResponse response) { + walletOnSolidity.futureGet(() -> super.doPost(request, response)); + } +} diff --git a/framework/src/main/java/org/tron/core/services/interfaceOnSolidity/http/solidity/HttpApiOnSolidityService.java b/framework/src/main/java/org/tron/core/services/interfaceOnSolidity/http/solidity/HttpApiOnSolidityService.java index b1d940ce2cd..f69597959f8 100644 --- a/framework/src/main/java/org/tron/core/services/interfaceOnSolidity/http/solidity/HttpApiOnSolidityService.java +++ b/framework/src/main/java/org/tron/core/services/interfaceOnSolidity/http/solidity/HttpApiOnSolidityService.java @@ -3,8 +3,6 @@ import java.util.EnumSet; import javax.servlet.DispatcherType; import lombok.extern.slf4j.Slf4j; -import org.eclipse.jetty.server.ConnectionLimit; -import org.eclipse.jetty.server.Server; import org.eclipse.jetty.servlet.FilterHolder; import org.eclipse.jetty.servlet.ServletContextHandler; import org.eclipse.jetty.servlet.ServletHolder; @@ -46,6 +44,7 @@ import org.tron.core.services.interfaceOnSolidity.http.GetNodeInfoOnSolidityServlet; import org.tron.core.services.interfaceOnSolidity.http.GetNowBlockOnSolidityServlet; import org.tron.core.services.interfaceOnSolidity.http.GetPaginatedAssetIssueListOnSolidityServlet; +import org.tron.core.services.interfaceOnSolidity.http.GetPaginatedNowWitnessListOnSolidityServlet; import org.tron.core.services.interfaceOnSolidity.http.GetRewardOnSolidityServlet; import org.tron.core.services.interfaceOnSolidity.http.GetTransactionCountByBlockNumOnSolidityServlet; import org.tron.core.services.interfaceOnSolidity.http.GetTransactionInfoByBlockNumOnSolidityServlet; @@ -60,7 +59,6 @@ import org.tron.core.services.interfaceOnSolidity.http.ScanShieldedTRC20NotesByOvkOnSolidityServlet; import org.tron.core.services.interfaceOnSolidity.http.TriggerConstantContractOnSolidityServlet; - @Slf4j(topic = "API") public class HttpApiOnSolidityService extends HttpService { @@ -74,6 +72,8 @@ public class HttpApiOnSolidityService extends HttpService { @Autowired private ListWitnessesOnSolidityServlet listWitnessesOnSolidityServlet; @Autowired + private GetPaginatedNowWitnessListOnSolidityServlet getPaginatedNowWitnessListOnSolidityServlet; + @Autowired private GetAssetIssueListOnSolidityServlet getAssetIssueListOnSolidityServlet; @Autowired private GetPaginatedAssetIssueListOnSolidityServlet getPaginatedAssetIssueListOnSolidityServlet; @@ -189,6 +189,8 @@ protected void addServlet(ServletContextHandler context) { context.addServlet(new ServletHolder(accountOnSolidityServlet), "/walletsolidity/getaccount"); context.addServlet(new ServletHolder(listWitnessesOnSolidityServlet), "/walletsolidity/listwitnesses"); + context.addServlet(new ServletHolder(getPaginatedNowWitnessListOnSolidityServlet), + "/walletsolidity/getpaginatednowwitnesslist"); context.addServlet(new ServletHolder(getAssetIssueListOnSolidityServlet), "/walletsolidity/getassetissuelist"); context.addServlet(new ServletHolder(getPaginatedAssetIssueListOnSolidityServlet), diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/JsonRpcApiUtil.java b/framework/src/main/java/org/tron/core/services/jsonrpc/JsonRpcApiUtil.java index 955ba55060f..4a60f14b534 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/JsonRpcApiUtil.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/JsonRpcApiUtil.java @@ -26,7 +26,7 @@ import org.tron.common.utils.Sha256Hash; import org.tron.common.utils.StringUtil; import org.tron.core.Wallet; -import org.tron.core.exception.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; import org.tron.protos.Protocol.Block; import org.tron.protos.Protocol.Transaction; import org.tron.protos.Protocol.Transaction.Contract.ContractType; diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/JsonRpcErrorResolver.java b/framework/src/main/java/org/tron/core/services/jsonrpc/JsonRpcErrorResolver.java new file mode 100644 index 00000000000..b92b3cf1af6 --- /dev/null +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/JsonRpcErrorResolver.java @@ -0,0 +1,81 @@ +package org.tron.core.services.jsonrpc; + +import com.fasterxml.jackson.databind.JsonNode; +import com.googlecode.jsonrpc4j.ErrorData; +import com.googlecode.jsonrpc4j.ErrorResolver; +import com.googlecode.jsonrpc4j.JsonRpcError; +import com.googlecode.jsonrpc4j.JsonRpcErrors; +import com.googlecode.jsonrpc4j.ReflectionUtil; +import java.lang.reflect.Method; +import java.util.List; +import org.tron.core.exception.jsonrpc.JsonRpcException; + +/** + * {@link ErrorResolver} that uses annotations. + */ +public enum JsonRpcErrorResolver implements ErrorResolver { + INSTANCE; + + /** + * {@inheritDoc} + */ + @Override + public JsonError resolveError( + Throwable thrownException, Method method, List arguments) { + JsonRpcError resolver = getResolverForException(thrownException, method); + if (notFoundResolver(resolver)) { + return null; + } + + String message = hasErrorMessage(resolver) ? resolver.message() : thrownException.getMessage(); + + // data priority: exception > annotation > default ErrorData + Object data = null; + if (thrownException instanceof JsonRpcException) { + JsonRpcException jsonRpcException = (JsonRpcException) thrownException; + data = jsonRpcException.getData(); + } + + if (data == null) { + data = hasErrorData(resolver) + ? resolver.data() + : new ErrorData(resolver.exception().getName(), message); + } + + return new JsonError(resolver.code(), message, data); + } + + private JsonRpcError getResolverForException(Throwable thrownException, Method method) { + JsonRpcErrors errors = ReflectionUtil.getAnnotation(method, JsonRpcErrors.class); + if (hasAnnotations(errors)) { + for (JsonRpcError errorDefined : errors.value()) { + if (isExceptionInstanceOfError(thrownException, errorDefined)) { + return errorDefined; + } + } + } + return null; + } + + private boolean notFoundResolver(JsonRpcError resolver) { + return resolver == null; + } + + private boolean hasErrorMessage(JsonRpcError em) { + // noinspection ConstantConditions + return em.message() != null && !em.message().trim().isEmpty(); + } + + private boolean hasErrorData(JsonRpcError em) { + // noinspection ConstantConditions + return em.data() != null && !em.data().trim().isEmpty(); + } + + private boolean hasAnnotations(JsonRpcErrors errors) { + return errors != null; + } + + private boolean isExceptionInstanceOfError(Throwable target, JsonRpcError em) { + return em.exception().isInstance(target); + } +} \ No newline at end of file diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/JsonRpcServlet.java b/framework/src/main/java/org/tron/core/services/jsonrpc/JsonRpcServlet.java index 878b71d86b5..104a0e9e470 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/JsonRpcServlet.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/JsonRpcServlet.java @@ -43,6 +43,7 @@ public void init(ServletConfig config) throws ServletException { true); rpcServer = new JsonRpcServer(compositeService); + rpcServer.setErrorResolver(JsonRpcErrorResolver.INSTANCE); HttpStatusCodeProvider httpStatusCodeProvider = new HttpStatusCodeProvider() { @Override diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/TronJsonRpc.java b/framework/src/main/java/org/tron/core/services/jsonrpc/TronJsonRpc.java index 52a3a2380d1..115df6ef9da 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/TronJsonRpc.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/TronJsonRpc.java @@ -7,6 +7,7 @@ import com.googlecode.jsonrpc4j.JsonRpcMethod; import java.io.IOException; import java.util.List; +import java.util.Objects; import java.util.concurrent.ExecutionException; import lombok.AllArgsConstructor; import lombok.Getter; @@ -18,17 +19,21 @@ import org.tron.common.utils.ByteArray; import org.tron.core.exception.BadItemException; import org.tron.core.exception.ItemNotFoundException; -import org.tron.core.exception.JsonRpcInternalException; -import org.tron.core.exception.JsonRpcInvalidParamsException; -import org.tron.core.exception.JsonRpcInvalidRequestException; -import org.tron.core.exception.JsonRpcMethodNotFoundException; -import org.tron.core.exception.JsonRpcTooManyResultException; +import org.tron.core.exception.jsonrpc.JsonRpcExceedLimitException; +import org.tron.core.exception.jsonrpc.JsonRpcInternalException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidRequestException; +import org.tron.core.exception.jsonrpc.JsonRpcMethodNotFoundException; +import org.tron.core.exception.jsonrpc.JsonRpcTooManyResultException; import org.tron.core.services.jsonrpc.types.BlockResult; import org.tron.core.services.jsonrpc.types.BuildArguments; import org.tron.core.services.jsonrpc.types.CallArguments; import org.tron.core.services.jsonrpc.types.TransactionReceipt; import org.tron.core.services.jsonrpc.types.TransactionResult; +/** + * Error code refers to https://www.quicknode.com/docs/ethereum/error-references + */ @Component public interface TronJsonRpc { @@ -146,6 +151,14 @@ TransactionResult getTransactionByBlockNumberAndIndex(String blockNumOrTag, Stri }) TransactionReceipt getTransactionReceipt(String txid) throws JsonRpcInvalidParamsException; + @JsonRpcMethod("eth_getBlockReceipts") + @JsonRpcErrors({ + @JsonRpcError(exception = JsonRpcInvalidParamsException.class, code = -32602, data = "{}"), + @JsonRpcError(exception = JsonRpcInternalException.class, code = -32000, data = "{}") + }) + List getBlockReceipts(String blockNumOrHashOrTag) + throws JsonRpcInvalidParamsException, JsonRpcInternalException; + @JsonRpcMethod("eth_call") @JsonRpcErrors({ @JsonRpcError(exception = JsonRpcInvalidRequestException.class, code = -32600, data = "{}"), @@ -284,9 +297,10 @@ String newFilter(FilterRequest fr) throws JsonRpcInvalidParamsException, @JsonRpcMethod("eth_newBlockFilter") @JsonRpcErrors({ + @JsonRpcError(exception = JsonRpcExceedLimitException.class, code = -32005, data = "{}"), @JsonRpcError(exception = JsonRpcMethodNotFoundException.class, code = -32601, data = "{}"), }) - String newBlockFilter() throws JsonRpcMethodNotFoundException; + String newBlockFilter() throws JsonRpcExceedLimitException, JsonRpcMethodNotFoundException; @JsonRpcMethod("eth_uninstallFilter") @JsonRpcErrors({ @@ -464,5 +478,35 @@ public LogFilterElement(String blockHash, Long blockNum, String txId, Integer tx } this.removed = removed; } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (o == null || this.getClass() != o.getClass()) { + return false; + } + LogFilterElement item = (LogFilterElement) o; + if (!Objects.equals(blockHash, item.blockHash)) { + return false; + } + if (!Objects.equals(transactionHash, item.transactionHash)) { + return false; + } + if (!Objects.equals(transactionIndex, item.transactionIndex)) { + return false; + } + if (!Objects.equals(logIndex, item.logIndex)) { + return false; + } + return removed == item.removed; + } + + @Override + public int hashCode() { + return Objects.hash(blockHash, transactionHash, transactionIndex, logIndex, removed); + } + } } diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/TronJsonRpcImpl.java b/framework/src/main/java/org/tron/core/services/jsonrpc/TronJsonRpcImpl.java index eb432432a1c..de939bdfff4 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/TronJsonRpcImpl.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/TronJsonRpcImpl.java @@ -11,10 +11,13 @@ import static org.tron.core.services.jsonrpc.JsonRpcApiUtil.triggerCallContract; import com.alibaba.fastjson.JSON; +import com.google.common.cache.Cache; +import com.google.common.cache.CacheBuilder; import com.google.protobuf.ByteString; import com.google.protobuf.GeneratedMessageV3; import java.io.Closeable; import java.io.IOException; +import java.util.ArrayList; import java.util.Arrays; import java.util.HashMap; import java.util.Iterator; @@ -24,11 +27,10 @@ import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ExecutionException; import java.util.concurrent.ExecutorService; -import java.util.regex.Matcher; +import java.util.concurrent.TimeUnit; import java.util.regex.Pattern; import lombok.Getter; import lombok.extern.slf4j.Slf4j; -import org.apache.commons.collections4.CollectionUtils; import org.apache.commons.lang3.StringUtils; import org.bouncycastle.util.encoders.Hex; import org.springframework.beans.factory.annotation.Autowired; @@ -38,6 +40,7 @@ import org.tron.api.GrpcAPI.Return; import org.tron.api.GrpcAPI.Return.response_code; import org.tron.api.GrpcAPI.TransactionExtention; +import org.tron.api.GrpcAPI.TransactionInfoList; import org.tron.common.crypto.Hash; import org.tron.common.es.ExecutorServiceManager; import org.tron.common.logsfilter.ContractEventParser; @@ -50,6 +53,7 @@ import org.tron.core.Wallet; import org.tron.core.capsule.BlockCapsule; import org.tron.core.capsule.TransactionCapsule; +import org.tron.core.config.args.Args; import org.tron.core.db.Manager; import org.tron.core.db2.core.Chainbase; import org.tron.core.exception.BadItemException; @@ -57,12 +61,13 @@ import org.tron.core.exception.ContractValidateException; import org.tron.core.exception.HeaderNotFound; import org.tron.core.exception.ItemNotFoundException; -import org.tron.core.exception.JsonRpcInternalException; -import org.tron.core.exception.JsonRpcInvalidParamsException; -import org.tron.core.exception.JsonRpcInvalidRequestException; -import org.tron.core.exception.JsonRpcMethodNotFoundException; -import org.tron.core.exception.JsonRpcTooManyResultException; import org.tron.core.exception.VMIllegalException; +import org.tron.core.exception.jsonrpc.JsonRpcExceedLimitException; +import org.tron.core.exception.jsonrpc.JsonRpcInternalException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidRequestException; +import org.tron.core.exception.jsonrpc.JsonRpcMethodNotFoundException; +import org.tron.core.exception.jsonrpc.JsonRpcTooManyResultException; import org.tron.core.services.NodeInfoService; import org.tron.core.services.http.JsonFormat; import org.tron.core.services.http.Util; @@ -76,12 +81,14 @@ import org.tron.core.services.jsonrpc.types.BuildArguments; import org.tron.core.services.jsonrpc.types.CallArguments; import org.tron.core.services.jsonrpc.types.TransactionReceipt; +import org.tron.core.services.jsonrpc.types.TransactionReceipt.TransactionContext; import org.tron.core.services.jsonrpc.types.TransactionResult; import org.tron.core.store.StorageRowStore; import org.tron.core.vm.program.Storage; import org.tron.program.Version; import org.tron.protos.Protocol.Account; import org.tron.protos.Protocol.Block; +import org.tron.protos.Protocol.ResourceReceipt; import org.tron.protos.Protocol.Transaction; import org.tron.protos.Protocol.Transaction.Contract.ContractType; import org.tron.protos.Protocol.Transaction.Result.code; @@ -94,6 +101,7 @@ import org.tron.protos.contract.SmartContractOuterClass.SmartContractDataWrapper; import org.tron.protos.contract.SmartContractOuterClass.TriggerSmartContract; + @Slf4j(topic = "API") @Component public class TronJsonRpcImpl implements TronJsonRpc, Closeable { @@ -106,6 +114,17 @@ public enum RequestSource { private static final String FILTER_NOT_FOUND = "filter not found"; public static final int EXPIRE_SECONDS = 5 * 60; + private static final int maxBlockFilterNum = Args.getInstance().getJsonRpcMaxBlockFilterNum(); + private static final Cache logElementCache = + CacheBuilder.newBuilder() + .maximumSize(300_000L) // 300s * tps(1000) * 1 log/tx ≈ 300_000 + .expireAfterWrite(EXPIRE_SECONDS, TimeUnit.SECONDS) + .recordStats().build(); //LRU cache + private static final Cache blockHashCache = + CacheBuilder.newBuilder() + .maximumSize(60_000L) // 300s * 200 block/s when syncing + .expireAfterWrite(EXPIRE_SECONDS, TimeUnit.SECONDS) + .recordStats().build(); //LRU cache /** * for log filter in Full Json-RPC */ @@ -177,16 +196,31 @@ public static void handleBLockFilter(BlockFilterCapsule blockFilterCapsule) { it = getBlockFilter2ResultFull().entrySet().iterator(); } + if (!it.hasNext()) { + return; + } + final String originalBlockHash = ByteArray.toJsonHex(blockFilterCapsule.getBlockHash()); + String cachedBlockHash; + try { + // compare with hashcode() first, then with equals(). If not exist, put it. + cachedBlockHash = blockHashCache.get(originalBlockHash, () -> originalBlockHash); + } catch (ExecutionException e) { + logger.error("Getting/loading blockHash from cache failed", e); // never happen + cachedBlockHash = originalBlockHash; + } while (it.hasNext()) { Entry entry = it.next(); if (entry.getValue().isExpire()) { it.remove(); continue; } - entry.getValue().getResult().add(ByteArray.toJsonHex(blockFilterCapsule.getBlockHash())); + entry.getValue().getResult().add(cachedBlockHash); } } + /** + * append LogsFilterCapsule's LogFilterElement list to each filter if matched + */ public static void handleLogsFilter(LogsFilterCapsule logsFilterCapsule) { Iterator> it; @@ -222,23 +256,28 @@ public static void handleLogsFilter(LogsFilterCapsule logsFilterCapsule) { LogMatch.matchBlock(logFilter, logsFilterCapsule.getBlockNumber(), logsFilterCapsule.getBlockHash(), logsFilterCapsule.getTxInfoList(), logsFilterCapsule.isRemoved()); - if (CollectionUtils.isNotEmpty(elements)) { - logFilterAndResult.getResult().addAll(elements); + + for (LogFilterElement element : elements) { + LogFilterElement cachedElement; + try { + // compare with hashcode() first, then with equals(). If not exist, put it. + cachedElement = logElementCache.get(element, () -> element); + } catch (ExecutionException e) { + logger.error("Getting/loading LogFilterElement from cache fails", e); // never happen + cachedElement = element; + } + logFilterAndResult.getResult().add(cachedElement); } } } @Override public String web3ClientVersion() { - Pattern shortVersion = Pattern.compile("(\\d\\.\\d).*"); - Matcher matcher = shortVersion.matcher(System.getProperty("java.version")); - matcher.matches(); - return String.join("/", Arrays.asList( "TRON", "v" + Version.getVersion(), System.getProperty("os.name"), - "Java" + matcher.group(1))); + "Java" + System.getProperty("java.specification.version"))); } @Override @@ -473,7 +512,6 @@ private String call(byte[] ownerAddressByte, byte[] contractAddressByte, long va } result = ByteArray.toJsonHex(listBytes); } else { - logger.error("trigger contract failed."); String errMsg = retBuilder.getMessage().toStringUtf8(); byte[] resData = trxExtBuilder.getConstantResult(0).toByteArray(); if (resData.length > 4 && Hex.toHexString(resData).startsWith(ERROR_SELECTOR)) { @@ -483,7 +521,12 @@ private String call(byte[] ownerAddressByte, byte[] contractAddressByte, long va errMsg += ": " + msg; } - throw new JsonRpcInternalException(errMsg); + if (resData.length > 0) { + throw new JsonRpcInternalException(errMsg, ByteArray.toJsonHex(resData)); + } else { + throw new JsonRpcInternalException(errMsg); + } + } return result; @@ -644,7 +687,12 @@ public String estimateGas(CallArguments args) throws JsonRpcInvalidRequestExcept errMsg += ": " + msg; } - throw new JsonRpcInternalException(errMsg); + if (data.length > 0) { + throw new JsonRpcInternalException(errMsg, ByteArray.toJsonHex(data)); + } else { + throw new JsonRpcInternalException(errMsg); + } + } else { if (supportEstimateEnergy) { @@ -763,6 +811,13 @@ public TransactionResult getTransactionByBlockNumberAndIndex(String blockNumOrTa return getTransactionByBlockAndIndex(block, index); } + /** + * Get a transaction receipt by transaction hash + * + * @param txId the transaction hash in hex format (with or without 0x prefix) + * @return TransactionReceipt object for the specified transaction, or null if not found + * @throws JsonRpcInvalidParamsException if the transaction hash format is invalid + */ @Override public TransactionReceipt getTransactionReceipt(String txId) throws JsonRpcInvalidParamsException { @@ -777,7 +832,126 @@ public TransactionReceipt getTransactionReceipt(String txId) return null; } - return new TransactionReceipt(block, transactionInfo, wallet); + BlockCapsule blockCapsule = new BlockCapsule(block); + long blockNum = blockCapsule.getNum(); + TransactionInfoList transactionInfoList = wallet.getTransactionInfoByBlockNum(blockNum); + long energyFee = wallet.getEnergyFee(blockCapsule.getTimeStamp()); + + // Find transaction context + TransactionReceipt.TransactionContext context + = findTransactionContext(transactionInfoList, + transactionInfo.getId()); + + if (context == null) { + return null; // Transaction not found in block + } + + return new TransactionReceipt(blockCapsule, transactionInfo, context, energyFee); + } + + /** + * Finds transaction context for a specific transaction ID within the block + * Calculates cumulative gas and log count up to the target transaction + * @param infoList the transactionInfo list for the block + * @param txId the transaction ID + * @return TransactionContext containing index and cumulative values, or null if not found + */ + private TransactionContext findTransactionContext(TransactionInfoList infoList, + ByteString txId) { + + long cumulativeGas = 0; + long cumulativeLogCount = 0; + + for (int index = 0; index < infoList.getTransactionInfoCount(); index++) { + TransactionInfo info = infoList.getTransactionInfo(index); + ResourceReceipt resourceReceipt = info.getReceipt(); + + if (info.getId().equals(txId)) { + return new TransactionContext(index, cumulativeGas, cumulativeLogCount); + } else { + cumulativeGas += resourceReceipt.getEnergyUsageTotal(); + cumulativeLogCount += info.getLogCount(); + } + } + return null; + } + + /** + * Get all transaction receipts for a specific block + * @param blockNumOrHashOrTag blockNumber or blockHash or tag, + * tag includes: latest, earliest, pending, finalized + * @return List of TransactionReceipt objects for all transactions in the block, + * null if block not found + * @throws JsonRpcInvalidParamsException if the parameter format is invalid + * @throws JsonRpcInternalException if there's an internal error + */ + @Override + public List getBlockReceipts(String blockNumOrHashOrTag) + throws JsonRpcInvalidParamsException, JsonRpcInternalException { + + Block block = null; + + if (Pattern.matches(HASH_REGEX, blockNumOrHashOrTag)) { + block = getBlockByJsonHash(blockNumOrHashOrTag); + } else { + block = wallet.getByJsonBlockId(blockNumOrHashOrTag); + } + + // block receipts not available: block is genesis, not produced yet, or pruned in light node + if (block == null || block.getBlockHeader().getRawData().getNumber() == 0) { + return null; + } + + BlockCapsule blockCapsule = new BlockCapsule(block); + long blockNum = blockCapsule.getNum(); + TransactionInfoList transactionInfoList = wallet.getTransactionInfoByBlockNum(blockNum); + + // energy price at the block timestamp + long energyFee = wallet.getEnergyFee(blockCapsule.getTimeStamp()); + + // Validate transaction list size consistency + int transactionSizeInBlock = blockCapsule.getTransactions().size(); + if (transactionSizeInBlock != transactionInfoList.getTransactionInfoCount()) { + throw new JsonRpcInternalException( + String.format("TransactionList size mismatch: " + + "block has %d transactions, but transactionInfoList has %d", + transactionSizeInBlock, transactionInfoList.getTransactionInfoCount())); + } + + return getTransactionReceiptsFromBlock(blockCapsule, transactionInfoList, energyFee); + } + + /** + * Get all TransactionReceipts from a block + * This method processes all transactions in the block + * and creates receipts with cumulative gas calculations + * @param blockCapsule the block containing transactions + * @param transactionInfoList the transaction info list for the block + * @param energyFee the energy price at the block timestamp + * @return List of TransactionReceipt objects for all transactions in the block + */ + private List getTransactionReceiptsFromBlock(BlockCapsule blockCapsule, + TransactionInfoList transactionInfoList, long energyFee) { + + List receipts = new ArrayList<>(); + long cumulativeGas = 0; + long cumulativeLogCount = 0; + + for (int index = 0; index < transactionInfoList.getTransactionInfoCount(); index++) { + TransactionInfo info = transactionInfoList.getTransactionInfo(index); + ResourceReceipt resourceReceipt = info.getReceipt(); + + TransactionReceipt.TransactionContext context = new TransactionContext( + index, cumulativeGas, cumulativeLogCount); + + // Use the constructor with pre-calculated context + TransactionReceipt receipt = new TransactionReceipt(blockCapsule, info, context, energyFee); + receipts.add(receipt); + + cumulativeGas += resourceReceipt.getEnergyUsageTotal(); + cumulativeLogCount += info.getLogCount(); + } + return receipts; } @Override @@ -1256,7 +1430,8 @@ public String newFilter(FilterRequest fr) throws JsonRpcInvalidParamsException, } @Override - public String newBlockFilter() throws JsonRpcMethodNotFoundException { + public String newBlockFilter() throws JsonRpcMethodNotFoundException, + JsonRpcExceedLimitException { disableInPBFT("eth_newBlockFilter"); Map blockFilter2Result; @@ -1265,6 +1440,10 @@ public String newBlockFilter() throws JsonRpcMethodNotFoundException { } else { blockFilter2Result = blockFilter2ResultSolidity; } + if (blockFilter2Result.size() >= maxBlockFilterNum) { + throw new JsonRpcExceedLimitException( + "exceed max block filters: " + maxBlockFilterNum + ", try again later"); + } BlockFilterAndResult filterAndResult = new BlockFilterAndResult(); String filterID = generateFilterId(); @@ -1394,6 +1573,8 @@ public static Object[] getFilterResult(String filterId, Map>> bitSetList = new ArrayList<>(); - + // 1. Collect all unique bitIndexes + Set uniqueBitIndexes = new HashSet<>(); for (int[] index : bitIndexes) { - List> futureList = new ArrayList<>(); - for (final int bitIndex : index) { //must be 3 - Future bitSetFuture = - sectionExecutor.submit(() -> sectionBloomStore.get(section, bitIndex)); - futureList.add(bitSetFuture); + for (int bitIndex : index) { //normally 3, but could be less due to hash collisions + uniqueBitIndexes.add(bitIndex); + } + } + + // 2. Submit concurrent requests for all unique bitIndexes + Map> bitIndexResults = new HashMap<>(); + for (int bitIndex : uniqueBitIndexes) { + Future future + = sectionExecutor.submit(() -> sectionBloomStore.get(section, bitIndex)); + bitIndexResults.put(bitIndex, future); + } + + // 3. Wait for all results and cache them + Map resultCache = new HashMap<>(); + for (Map.Entry> entry : bitIndexResults.entrySet()) { + BitSet result = entry.getValue().get(); + if (result != null) { + resultCache.put(entry.getKey(), result); } - bitSetList.add(futureList); } - BitSet bitSet = new BitSet(SectionBloomStore.BLOCK_PER_SECTION); - for (List> futureList : bitSetList) { - // initial a BitSet with all 1 - BitSet subBitSet = new BitSet(SectionBloomStore.BLOCK_PER_SECTION); - subBitSet.set(0, SectionBloomStore.BLOCK_PER_SECTION); + // 4. Process valid groups with reused BitSet objects + BitSet finalResult = new BitSet(SectionBloomStore.BLOCK_PER_SECTION); + BitSet tempBitSet = new BitSet(SectionBloomStore.BLOCK_PER_SECTION); + + for (int[] index : bitIndexes) { + + // init tempBitSet with all 1 + tempBitSet.set(0, SectionBloomStore.BLOCK_PER_SECTION); + // and condition in second dimension - for (Future future : futureList) { - BitSet one = future.get(); - if (one == null) { //match nothing - subBitSet.clear(); + for (int bitIndex : index) { + BitSet cached = resultCache.get(bitIndex); + if (cached == null) { //match nothing + tempBitSet.clear(); break; } // "and" condition in second dimension - subBitSet.and(one); + tempBitSet.and(cached); + if (tempBitSet.isEmpty()) { + break; + } } + // "or" condition in first dimension - bitSet.or(subBitSet); + if (!tempBitSet.isEmpty()) { + finalResult.or(tempBitSet); + } } - return bitSet; + + return finalResult; } /** diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilter.java b/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilter.java index ce315e506d2..42bc123d4bc 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilter.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilter.java @@ -14,7 +14,7 @@ import org.tron.common.crypto.Hash; import org.tron.common.runtime.vm.DataWord; import org.tron.core.config.args.Args; -import org.tron.core.exception.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; import org.tron.core.services.jsonrpc.TronJsonRpc.FilterRequest; import org.tron.protos.Protocol.TransactionInfo.Log; diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilterAndResult.java b/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilterAndResult.java index 3b893aec4cf..57739819d1e 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilterAndResult.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilterAndResult.java @@ -5,7 +5,7 @@ import java.util.concurrent.LinkedBlockingQueue; import lombok.Getter; import org.tron.core.Wallet; -import org.tron.core.exception.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; import org.tron.core.services.jsonrpc.TronJsonRpc.FilterRequest; import org.tron.core.services.jsonrpc.TronJsonRpc.LogFilterElement; diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilterWrapper.java b/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilterWrapper.java index c0cd1ff12df..97a012b7f9a 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilterWrapper.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogFilterWrapper.java @@ -8,7 +8,7 @@ import org.tron.common.utils.ByteArray; import org.tron.core.Wallet; import org.tron.core.config.args.Args; -import org.tron.core.exception.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; import org.tron.core.services.jsonrpc.JsonRpcApiUtil; import org.tron.core.services.jsonrpc.TronJsonRpc.FilterRequest; import org.tron.protos.Protocol.Block; diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogMatch.java b/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogMatch.java index d96aa07f9a4..cf958d1e2cb 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogMatch.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/filters/LogMatch.java @@ -10,7 +10,7 @@ import org.tron.core.db.Manager; import org.tron.core.exception.BadItemException; import org.tron.core.exception.ItemNotFoundException; -import org.tron.core.exception.JsonRpcTooManyResultException; +import org.tron.core.exception.jsonrpc.JsonRpcTooManyResultException; import org.tron.core.services.jsonrpc.TronJsonRpc.LogFilterElement; import org.tron.protos.Protocol.TransactionInfo; import org.tron.protos.Protocol.TransactionInfo.Log; @@ -83,17 +83,13 @@ public LogFilterElement[] matchBlockOneByOne() List logFilterElementList = new ArrayList<>(); for (long blockNum : blockNumList) { - TransactionRetCapsule transactionRetCapsule = - manager.getTransactionRetStore() - .getTransactionInfoByBlockNum(ByteArray.fromLong(blockNum)); - if (transactionRetCapsule == null) { - //if query condition (address and topics) is empty, we will traversal every block, - //include empty block + List transactionInfoList = + manager.getTransactionInfoByBlockNum(blockNum).getTransactionInfoList(); + //if query condition (address and topics) is empty, we will traversal every block, + //include empty block + if (transactionInfoList.isEmpty()) { continue; } - TransactionRet transactionRet = transactionRetCapsule.getInstance(); - List transactionInfoList = transactionRet.getTransactioninfoList(); - String blockHash = manager.getChainBaseManager().getBlockIdByNum(blockNum).toString(); List matchedLog = matchBlock(logFilterWrapper.getLogFilter(), blockNum, blockHash, transactionInfoList, false); diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/types/BuildArguments.java b/framework/src/main/java/org/tron/core/services/jsonrpc/types/BuildArguments.java index 223e807e622..490219a13d9 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/types/BuildArguments.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/types/BuildArguments.java @@ -14,8 +14,8 @@ import org.apache.commons.lang3.StringUtils; import org.tron.api.GrpcAPI.BytesMessage; import org.tron.core.Wallet; -import org.tron.core.exception.JsonRpcInvalidParamsException; -import org.tron.core.exception.JsonRpcInvalidRequestException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidRequestException; import org.tron.protos.Protocol.Transaction.Contract.ContractType; import org.tron.protos.contract.SmartContractOuterClass.SmartContract; diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/types/CallArguments.java b/framework/src/main/java/org/tron/core/services/jsonrpc/types/CallArguments.java index 1485448c4b6..70edd1ad94f 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/types/CallArguments.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/types/CallArguments.java @@ -13,8 +13,8 @@ import org.apache.commons.lang3.StringUtils; import org.tron.api.GrpcAPI.BytesMessage; import org.tron.core.Wallet; -import org.tron.core.exception.JsonRpcInvalidParamsException; -import org.tron.core.exception.JsonRpcInvalidRequestException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidRequestException; import org.tron.protos.Protocol.Transaction.Contract.ContractType; import org.tron.protos.contract.SmartContractOuterClass.SmartContract; diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/types/TransactionReceipt.java b/framework/src/main/java/org/tron/core/services/jsonrpc/types/TransactionReceipt.java index 81b7c763cca..fd57ec0d9ad 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/types/TransactionReceipt.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/types/TransactionReceipt.java @@ -9,179 +9,151 @@ import java.util.List; import lombok.Getter; import lombok.Setter; -import org.tron.api.GrpcAPI.TransactionInfoList; import org.tron.common.utils.ByteArray; -import org.tron.core.Wallet; import org.tron.core.capsule.BlockCapsule; import org.tron.core.capsule.TransactionCapsule; import org.tron.protos.Protocol; -import org.tron.protos.Protocol.ResourceReceipt; import org.tron.protos.Protocol.Transaction.Contract; import org.tron.protos.Protocol.Transaction.Contract.ContractType; import org.tron.protos.Protocol.TransactionInfo; +@Getter +@Setter @JsonPropertyOrder(alphabetic = true) public class TransactionReceipt { - @JsonPropertyOrder(alphabetic = true) + @Getter + @Setter public static class TransactionLog { - @Getter - @Setter private String logIndex; - @Getter - @Setter private String blockHash; - @Getter - @Setter private String blockNumber; - @Getter - @Setter private String transactionIndex; - @Getter - @Setter private String transactionHash; - @Getter - @Setter private String address; - @Getter - @Setter private String data; - @Getter - @Setter private String[] topics; - @Getter - @Setter private boolean removed = false; - public TransactionLog() { - } + public TransactionLog() {} } - @Getter - @Setter private String blockHash; - @Getter - @Setter private String blockNumber; - @Getter - @Setter private String transactionIndex; - @Getter - @Setter private String transactionHash; - @Getter - @Setter private String from; - @Getter - @Setter private String to; - @Getter - @Setter private String cumulativeGasUsed; - @Getter - @Setter private String effectiveGasPrice; - @Getter - @Setter private String gasUsed; - @Getter - @Setter private String contractAddress; - @Getter - @Setter private TransactionLog[] logs; - @Getter - @Setter - private String logsBloom; - @JsonInclude(JsonInclude.Include.NON_NULL) - public String root; // 32 bytes of post-transaction stateroot (pre Byzantium) - @JsonInclude(JsonInclude.Include.NON_NULL) - public String status; // either 1 (success) or 0 (failure) (post Byzantium) + private String logsBloom = ByteArray.toJsonHex(new byte[256]); // default no value; - @Getter - @Setter - private String type = "0x0"; - - public TransactionReceipt(Protocol.Block block, TransactionInfo txInfo, Wallet wallet) { - BlockCapsule blockCapsule = new BlockCapsule(block); - String txid = ByteArray.toHexString(txInfo.getId().toByteArray()); - long blockNum = blockCapsule.getNum(); - - Protocol.Transaction transaction = null; - long cumulativeGas = 0; - long cumulativeLogCount = 0; - - TransactionInfoList infoList = wallet.getTransactionInfoByBlockNum(blockNum); - for (int index = 0; index < infoList.getTransactionInfoCount(); index++) { - TransactionInfo info = infoList.getTransactionInfo(index); - ResourceReceipt resourceReceipt = info.getReceipt(); - - long energyUsage = resourceReceipt.getEnergyUsageTotal(); - cumulativeGas += energyUsage; - - if (ByteArray.toHexString(info.getId().toByteArray()).equals(txid)) { - transactionIndex = ByteArray.toJsonHex(index); - cumulativeGasUsed = ByteArray.toJsonHex(cumulativeGas); - gasUsed = ByteArray.toJsonHex(energyUsage); - status = resourceReceipt.getResultValue() <= 1 ? "0x1" : "0x0"; - - transaction = block.getTransactions(index); - break; - } else { - cumulativeLogCount += info.getLogCount(); - } - } - - blockHash = ByteArray.toJsonHex(blockCapsule.getBlockId().getBytes()); - blockNumber = ByteArray.toJsonHex(blockCapsule.getNum()); - transactionHash = ByteArray.toJsonHex(txInfo.getId().toByteArray()); - effectiveGasPrice = ByteArray.toJsonHex(wallet.getEnergyFee(blockCapsule.getTimeStamp())); - - from = null; - to = null; - contractAddress = null; + @JsonInclude(JsonInclude.Include.NON_NULL) + private String root = null; // 32 bytes of post-transaction stateroot (pre Byzantium) - if (transaction != null && !transaction.getRawData().getContractList().isEmpty()) { + @JsonInclude(JsonInclude.Include.NON_NULL) + private String status; // either 1 (success) or 0 (failure) (post Byzantium) + + private String type = "0x0"; // legacy transaction, set 0 in java-tron + + /** + * Constructor for creating a TransactionReceipt + * + * @param blockCapsule the block containing the transaction + * @param txInfo the transaction info containing execution details + * @param context the pre-calculated transaction context + * @param energyFee the energy price at the block timestamp + */ + public TransactionReceipt( + BlockCapsule blockCapsule, + TransactionInfo txInfo, + TransactionContext context, + long energyFee) { + // Set basic fields + this.blockHash = ByteArray.toJsonHex(blockCapsule.getBlockId().getBytes()); + this.blockNumber = ByteArray.toJsonHex(blockCapsule.getNum()); + this.transactionHash = ByteArray.toJsonHex(txInfo.getId().toByteArray()); + this.transactionIndex = ByteArray.toJsonHex(context.index); + // Compute cumulative gas until this transaction + this.cumulativeGasUsed = + ByteArray.toJsonHex(context.cumulativeGas + txInfo.getReceipt().getEnergyUsageTotal()); + this.gasUsed = ByteArray.toJsonHex(txInfo.getReceipt().getEnergyUsageTotal()); + this.status = txInfo.getReceipt().getResultValue() <= 1 ? "0x1" : "0x0"; + this.effectiveGasPrice = ByteArray.toJsonHex(energyFee); + + // Set contract fields + this.from = null; + this.to = null; + this.contractAddress = null; + + TransactionCapsule txCapsule = blockCapsule.getTransactions().get(context.index); + Protocol.Transaction transaction = txCapsule.getInstance(); + if (!transaction.getRawData().getContractList().isEmpty()) { Contract contract = transaction.getRawData().getContract(0); byte[] fromByte = TransactionCapsule.getOwner(contract); byte[] toByte = getToAddress(transaction); - from = ByteArray.toJsonHexAddress(fromByte); - to = ByteArray.toJsonHexAddress(toByte); + this.from = ByteArray.toJsonHexAddress(fromByte); + this.to = ByteArray.toJsonHexAddress(toByte); if (contract.getType() == ContractType.CreateSmartContract) { - contractAddress = ByteArray.toJsonHexAddress(txInfo.getContractAddress().toByteArray()); + this.contractAddress = + ByteArray.toJsonHexAddress(txInfo.getContractAddress().toByteArray()); } } - // logs + // Set logs List logList = new ArrayList<>(); - for (int index = 0; index < txInfo.getLogCount(); index++) { - TransactionInfo.Log log = txInfo.getLogList().get(index); - - TransactionReceipt.TransactionLog transactionLog = new TransactionReceipt.TransactionLog(); - // index is the index in the block - transactionLog.logIndex = ByteArray.toJsonHex(index + cumulativeLogCount); - transactionLog.transactionHash = transactionHash; - transactionLog.transactionIndex = transactionIndex; - transactionLog.blockHash = blockHash; - transactionLog.blockNumber = blockNumber; + for (int logIndex = 0; logIndex < txInfo.getLogCount(); logIndex++) { + TransactionInfo.Log log = txInfo.getLogList().get(logIndex); + TransactionLog transactionLog = new TransactionLog(); + transactionLog.setLogIndex(ByteArray.toJsonHex(logIndex + context.cumulativeLogCount)); + transactionLog.setTransactionHash(this.transactionHash); + transactionLog.setTransactionIndex(this.transactionIndex); + transactionLog.setBlockHash(this.blockHash); + transactionLog.setBlockNumber(this.blockNumber); + byte[] addressByte = convertToTronAddress(log.getAddress().toByteArray()); - transactionLog.address = ByteArray.toJsonHexAddress(addressByte); - transactionLog.data = ByteArray.toJsonHex(log.getData().toByteArray()); + transactionLog.setAddress(ByteArray.toJsonHexAddress(addressByte)); + transactionLog.setData(ByteArray.toJsonHex(log.getData().toByteArray())); String[] topics = new String[log.getTopicsCount()]; for (int i = 0; i < log.getTopicsCount(); i++) { topics[i] = ByteArray.toJsonHex(log.getTopics(i).toByteArray()); } - transactionLog.topics = topics; + transactionLog.setTopics(topics); logList.add(transactionLog); } + this.logs = logList.toArray(new TransactionLog[0]); + + } - logs = logList.toArray(new TransactionReceipt.TransactionLog[logList.size()]); - logsBloom = ByteArray.toJsonHex(new byte[256]); // no value - root = null; + /** + * Context class to hold transaction creation parameters Contains index and cumulative values + * needed for receipt creation + */ + @Getter + public static class TransactionContext { + private final int index; + private final long cumulativeGas; + private final long cumulativeLogCount; + + /** + * Creates a transaction context with the given parameters + * + * @param index the transaction index within the block + * @param cumulativeGas the cumulative gas used up to this transaction + * @param cumulativeLogCount the cumulative log count up to this transaction + */ + public TransactionContext(int index, long cumulativeGas, long cumulativeLogCount) { + this.index = index; + this.cumulativeGas = cumulativeGas; + this.cumulativeLogCount = cumulativeLogCount; + } } -} \ No newline at end of file +} diff --git a/framework/src/main/java/org/tron/core/services/jsonrpc/types/TransactionResult.java b/framework/src/main/java/org/tron/core/services/jsonrpc/types/TransactionResult.java index 389c58505cd..57650355d46 100644 --- a/framework/src/main/java/org/tron/core/services/jsonrpc/types/TransactionResult.java +++ b/framework/src/main/java/org/tron/core/services/jsonrpc/types/TransactionResult.java @@ -5,9 +5,9 @@ import com.fasterxml.jackson.annotation.JsonPropertyOrder; import com.google.protobuf.ByteString; -import java.util.Arrays; import lombok.Getter; import lombok.ToString; +import org.tron.common.crypto.Rsv; import org.tron.common.utils.ByteArray; import org.tron.core.Wallet; import org.tron.core.capsule.BlockCapsule; @@ -65,16 +65,10 @@ private void parseSignature(Transaction tx) { } ByteString signature = tx.getSignature(0); // r[32] + s[32] + v[1] - byte[] signData = signature.toByteArray(); - byte[] rByte = Arrays.copyOfRange(signData, 0, 32); - byte[] sByte = Arrays.copyOfRange(signData, 32, 64); - byte vByte = signData[64]; - if (vByte < 27) { - vByte += 27; - } - v = ByteArray.toJsonHex(vByte); - r = ByteArray.toJsonHex(rByte); - s = ByteArray.toJsonHex(sByte); + Rsv rsv = Rsv.fromSignature(signature.toByteArray()); + r = ByteArray.toJsonHex(rsv.getR()); + s = ByteArray.toJsonHex(rsv.getS()); + v = ByteArray.toJsonHex(rsv.getV()); } private String parseInput(Transaction tx) { diff --git a/framework/src/main/java/org/tron/core/zen/ZksnarkInitService.java b/framework/src/main/java/org/tron/core/zen/ZksnarkInitService.java index a1f812426f7..dfc4b428836 100644 --- a/framework/src/main/java/org/tron/core/zen/ZksnarkInitService.java +++ b/framework/src/main/java/org/tron/core/zen/ZksnarkInitService.java @@ -3,6 +3,7 @@ import java.io.File; import java.io.IOException; import java.io.InputStream; +import java.util.concurrent.atomic.AtomicBoolean; import javax.annotation.PostConstruct; import lombok.extern.slf4j.Slf4j; import org.apache.commons.io.FileUtils; @@ -16,34 +17,43 @@ @Component public class ZksnarkInitService { + private static final AtomicBoolean initialized = new AtomicBoolean(false); + @PostConstruct private void init() { librustzcashInitZksnarkParams(); } public static void librustzcashInitZksnarkParams() { - logger.info("init zk param begin"); - - if (!JLibrustzcash.isOpenZen()) { - logger.info("zen switch is off, zen will not start."); + if (initialized.get()) { + logger.info("zk param already initialized"); return; } - String spendPath = getParamsFile("sapling-spend.params"); - String spendHash = "25fd9a0d1c1be0526c14662947ae95b758fe9f3d7fb7f55e9b4437830dcc6215a7ce3ea465" - + "914b157715b7a4d681389ea4aa84438190e185d5e4c93574d3a19a"; + synchronized (ZksnarkInitService.class) { + if (initialized.get()) { + logger.info("zk param already initialized"); + return; + } + logger.info("init zk param begin"); - String outputPath = getParamsFile("sapling-output.params"); - String outputHash = "a1cb23b93256adce5bce2cb09cefbc96a1d16572675ceb691e9a3626ec15b5b546926ff1c" - + "536cfe3a9df07d796b32fdfc3e5d99d65567257bf286cd2858d71a6"; + String spendPath = getParamsFile("sapling-spend.params"); + String spendHash = "25fd9a0d1c1be0526c14662947ae95b758fe9f3d7fb7f55e9b4437830dcc6215a7ce3ea" + + "465914b157715b7a4d681389ea4aa84438190e185d5e4c93574d3a19a"; - try { - JLibrustzcash.librustzcashInitZksnarkParams( - new LibrustzcashParam.InitZksnarkParams(spendPath, spendHash, outputPath, outputHash)); - } catch (ZksnarkException e) { - throw new TronError(e, TronError.ErrCode.ZCASH_INIT); + String outputPath = getParamsFile("sapling-output.params"); + String outputHash = "a1cb23b93256adce5bce2cb09cefbc96a1d16572675ceb691e9a3626ec15b5b546926f" + + "f1c536cfe3a9df07d796b32fdfc3e5d99d65567257bf286cd2858d71a6"; + + try { + JLibrustzcash.librustzcashInitZksnarkParams( + new LibrustzcashParam.InitZksnarkParams(spendPath, spendHash, outputPath, outputHash)); + } catch (ZksnarkException e) { + throw new TronError(e, TronError.ErrCode.ZCASH_INIT); + } + initialized.set(true); + logger.info("init zk param done"); } - logger.info("init zk param done"); } private static String getParamsFile(String fileName) { @@ -61,4 +71,4 @@ private static String getParamsFile(String fileName) { } return fileOut.getAbsolutePath(); } -} \ No newline at end of file +} diff --git a/framework/src/main/java/org/tron/program/DBConvert.java b/framework/src/main/java/org/tron/program/DBConvert.java deleted file mode 100644 index 7b9d63544dc..00000000000 --- a/framework/src/main/java/org/tron/program/DBConvert.java +++ /dev/null @@ -1,413 +0,0 @@ -package org.tron.program; - -import static org.fusesource.leveldbjni.JniDBFactory.factory; -import static org.tron.common.math.Maths.max; - -import java.io.File; -import java.nio.file.Path; -import java.nio.file.Paths; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; -import java.util.Map; -import java.util.Objects; -import java.util.concurrent.ArrayBlockingQueue; -import java.util.concurrent.Callable; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.Executors; -import java.util.concurrent.Future; -import java.util.concurrent.ThreadPoolExecutor; -import java.util.concurrent.TimeUnit; -import java.util.stream.Collectors; -import lombok.extern.slf4j.Slf4j; -import org.fusesource.leveldbjni.JniDBFactory; -import org.iq80.leveldb.CompressionType; -import org.iq80.leveldb.DB; -import org.iq80.leveldb.DBIterator; -import org.rocksdb.BlockBasedTableConfig; -import org.rocksdb.BloomFilter; -import org.rocksdb.ComparatorOptions; -import org.rocksdb.Options; -import org.rocksdb.RocksDB; -import org.rocksdb.RocksDBException; -import org.rocksdb.RocksIterator; -import org.rocksdb.Status; -import org.tron.common.utils.FileUtil; -import org.tron.common.utils.MarketOrderPriceComparatorForLevelDB; -import org.tron.common.utils.MarketOrderPriceComparatorForRockDB; -import org.tron.common.utils.PropUtil; - -@Slf4j -public class DBConvert implements Callable { - - static { - RocksDB.loadLibrary(); - } - - private final String srcDir; - private final String dstDir; - private final String dbName; - private final Path srcDbPath; - private final Path dstDbPath; - - private long srcDbKeyCount = 0L; - private long dstDbKeyCount = 0L; - private long srcDbKeySum = 0L; - private long dstDbKeySum = 0L; - private long srcDbValueSum = 0L; - private long dstDbValueSum = 0L; - private final long startTime; - private static final int CPUS = Runtime.getRuntime().availableProcessors(); - private static final int BATCH = 256; - private static final String CHECKPOINT_V2_DIR_NAME = "checkpoint"; - - - @Override - public Boolean call() throws Exception { - return doConvert(); - } - - public DBConvert(String src, String dst, String name) { - this.srcDir = src; - this.dstDir = dst; - this.dbName = name; - this.srcDbPath = Paths.get(this.srcDir, name); - this.dstDbPath = Paths.get(this.dstDir, name); - this.startTime = System.currentTimeMillis(); - } - - public static org.iq80.leveldb.Options newDefaultLevelDbOptions() { - org.iq80.leveldb.Options dbOptions = new org.iq80.leveldb.Options(); - dbOptions.createIfMissing(true); - dbOptions.paranoidChecks(true); - dbOptions.verifyChecksums(true); - dbOptions.compressionType(CompressionType.SNAPPY); - dbOptions.blockSize(4 * 1024); - dbOptions.writeBufferSize(10 * 1024 * 1024); - dbOptions.cacheSize(10 * 1024 * 1024L); - dbOptions.maxOpenFiles(1000); - return dbOptions; - } - - public static void main(String[] args) { - int code = run(args); - logger.info("exit code {}.", code); - System.out.printf("exit code %d.\n", code); - System.exit(code); - } - - public static int run(String[] args) { - String dbSrc; - String dbDst; - if (args.length < 2) { - dbSrc = "output-directory/database"; - dbDst = "output-directory-dst/database"; - } else { - dbSrc = args[0]; - dbDst = args[1]; - } - File dbDirectory = new File(dbSrc); - if (!dbDirectory.exists()) { - logger.info(" {} does not exist.", dbSrc); - return 404; - } - List files = Arrays.stream(Objects.requireNonNull(dbDirectory.listFiles())) - .filter(File::isDirectory) - .filter(e -> !CHECKPOINT_V2_DIR_NAME.equals(e.getName())) - .collect(Collectors.toList()); - - // add checkpoint v2 convert - File cpV2Dir = new File(Paths.get(dbSrc, CHECKPOINT_V2_DIR_NAME).toString()); - List cpList = null; - if (cpV2Dir.exists()) { - cpList = Arrays.stream(Objects.requireNonNull(cpV2Dir.listFiles())) - .filter(File::isDirectory) - .collect(Collectors.toList()); - } - - if (files.isEmpty()) { - logger.info("{} does not contain any database.", dbSrc); - return 0; - } - final long time = System.currentTimeMillis(); - final List> res = new ArrayList<>(); - - final ThreadPoolExecutor esDb = new ThreadPoolExecutor( - CPUS, 16 * CPUS, 1, TimeUnit.MINUTES, - new ArrayBlockingQueue<>(CPUS, true), Executors.defaultThreadFactory(), - new ThreadPoolExecutor.CallerRunsPolicy()); - - esDb.allowCoreThreadTimeOut(true); - - files.forEach(f -> res.add(esDb.submit(new DBConvert(dbSrc, dbDst, f.getName())))); - // convert v2 - if (cpList != null) { - cpList.forEach(f -> res.add(esDb.submit( - new DBConvert(dbSrc + "/" + CHECKPOINT_V2_DIR_NAME, - dbDst + "/" + CHECKPOINT_V2_DIR_NAME, f.getName())))); - } - - int fails = res.size(); - - for (Future re : res) { - try { - if (re.get()) { - fails--; - } - } catch (InterruptedException e) { - logger.error("{}", e); - Thread.currentThread().interrupt(); - } catch (ExecutionException e) { - logger.error("{}", e); - } - } - - esDb.shutdown(); - logger.info("database convert use {} seconds total.", - (System.currentTimeMillis() - time) / 1000); - if (fails > 0) { - logger.error("failed!!!!!!!!!!!!!!!!!!!!!!!! size:{}", fails); - } - return fails; - } - - public DB newLevelDb(Path db) throws Exception { - DB database; - File file = db.toFile(); - org.iq80.leveldb.Options dbOptions = newDefaultLevelDbOptions(); - if ("market_pair_price_to_order".equalsIgnoreCase(this.dbName)) { - dbOptions.comparator(new MarketOrderPriceComparatorForLevelDB()); - } - database = factory.open(file, dbOptions); - return database; - } - - private Options newDefaultRocksDbOptions() { - Options options = new Options(); - options.setCreateIfMissing(true); - options.setIncreaseParallelism(1); - options.setNumLevels(7); - options.setMaxOpenFiles(5000); - options.setTargetFileSizeBase(64 * 1024 * 1024); - options.setTargetFileSizeMultiplier(1); - options.setMaxBytesForLevelBase(512 * 1024 * 1024); - options.setMaxBackgroundCompactions(max(1, Runtime.getRuntime().availableProcessors(), true)); - options.setLevel0FileNumCompactionTrigger(4); - options.setLevelCompactionDynamicLevelBytes(true); - if ("market_pair_price_to_order".equalsIgnoreCase(this.dbName)) { - options.setComparator(new MarketOrderPriceComparatorForRockDB(new ComparatorOptions())); - } - final BlockBasedTableConfig tableCfg; - options.setTableFormatConfig(tableCfg = new BlockBasedTableConfig()); - tableCfg.setBlockSize(64 * 1024); - tableCfg.setBlockCacheSize(32 * 1024 * 1024); - tableCfg.setCacheIndexAndFilterBlocks(true); - tableCfg.setPinL0FilterAndIndexBlocksInCache(true); - tableCfg.setFilter(new BloomFilter(10, false)); - options.prepareForBulkLoad(); - return options; - } - - public RocksDB newRocksDb(Path db) { - RocksDB database = null; - try (Options options = newDefaultRocksDbOptions()) { - database = RocksDB.open(options, db.toString()); - } catch (Exception e) { - logger.error("{}", e); - } - return database; - } - - private void batchInsert(RocksDB rocks, List keys, List values) - throws Exception { - try (org.rocksdb.WriteBatch batch = new org.rocksdb.WriteBatch()) { - for (int i = 0; i < keys.size(); i++) { - byte[] k = keys.get(i); - byte[] v = values.get(i); - batch.put(k, v); - } - write(rocks, batch); - } - keys.clear(); - values.clear(); - } - - /** - * https://github.com/facebook/rocksdb/issues/6625 - * @param rocks db - * @param batch write batch - * @throws Exception RocksDBException - */ - private void write(RocksDB rocks, org.rocksdb.WriteBatch batch) throws Exception { - try { - rocks.write(new org.rocksdb.WriteOptions(), batch); - } catch (RocksDBException e) { - // retry - if (maybeRetry(e)) { - TimeUnit.MILLISECONDS.sleep(1); - write(rocks, batch); - } else { - throw e; - } - } - } - - private boolean maybeRetry(RocksDBException e) { - boolean retry = false; - if (e.getStatus() != null) { - retry = e.getStatus().getCode() == Status.Code.TryAgain - || e.getStatus().getCode() == Status.Code.Busy - || e.getStatus().getCode() == Status.Code.Incomplete; - } - return retry || (e.getMessage() != null && ("Write stall".equalsIgnoreCase(e.getMessage()) - || ("Incomplete").equalsIgnoreCase(e.getMessage()))); - } - - /** - * https://github.com/facebook/rocksdb/wiki/RocksDB-FAQ . - * What's the fastest way to load data into RocksDB? - * @param level leveldb - * @param rocks rocksdb - * @return if ok - */ - public boolean convertLevelToRocksBatchIterator(DB level, RocksDB rocks) { - // convert - List keys = new ArrayList<>(BATCH); - List values = new ArrayList<>(BATCH); - try (DBIterator levelIterator = level.iterator( - new org.iq80.leveldb.ReadOptions().fillCache(false))) { - - JniDBFactory.pushMemoryPool(1024 * 1024); - levelIterator.seekToFirst(); - - while (levelIterator.hasNext()) { - Map.Entry entry = levelIterator.next(); - byte[] key = entry.getKey(); - byte[] value = entry.getValue(); - srcDbKeyCount++; - srcDbKeySum = byteArrayToIntWithOne(srcDbKeySum, key); - srcDbValueSum = byteArrayToIntWithOne(srcDbValueSum, value); - keys.add(key); - values.add(value); - if (keys.size() >= BATCH) { - try { - batchInsert(rocks, keys, values); - } catch (Exception e) { - logger.error("{}", e); - return false; - } - } - } - - if (!keys.isEmpty()) { - try { - batchInsert(rocks, keys, values); - } catch (Exception e) { - logger.error("{}", e); - return false; - } - } - // check - check(rocks); - } catch (Exception e) { - logger.error("{}", e); - return false; - } finally { - try { - level.close(); - rocks.close(); - JniDBFactory.popMemoryPool(); - } catch (Exception e1) { - logger.error("{}", e1); - } - } - return dstDbKeyCount == srcDbKeyCount && dstDbKeySum == srcDbKeySum - && dstDbValueSum == srcDbValueSum; - } - - private void check(RocksDB rocks) throws RocksDBException { - logger.info("check database {} start", this.dbName); - // manually call CompactRange() - logger.info("compact database {} start", this.dbName); - rocks.compactRange(); - logger.info("compact database {} end", this.dbName); - // check - try (org.rocksdb.ReadOptions r = new org.rocksdb.ReadOptions().setFillCache(false); - RocksIterator rocksIterator = rocks.newIterator(r)) { - for (rocksIterator.seekToFirst(); rocksIterator.isValid(); rocksIterator.next()) { - byte[] key = rocksIterator.key(); - byte[] value = rocksIterator.value(); - dstDbKeyCount++; - dstDbKeySum = byteArrayToIntWithOne(dstDbKeySum, key); - dstDbValueSum = byteArrayToIntWithOne(dstDbValueSum, value); - } - } - logger.info("check database {} end", this.dbName); - } - - public boolean createEngine(String dir) { - String enginePath = dir + File.separator + "engine.properties"; - - if (!FileUtil.createFileIfNotExists(enginePath)) { - return false; - } - - return PropUtil.writeProperty(enginePath, "ENGINE", "ROCKSDB"); - } - - public boolean checkDone(String dir) { - String enginePath = dir + File.separator + "engine.properties"; - return FileUtil.isExists(enginePath); - - } - - public boolean doConvert() throws Exception { - - if (checkDone(this.dstDbPath.toString())) { - logger.info(" {} is done, skip it.", this.dbName); - return true; - } - - File levelDbFile = srcDbPath.toFile(); - if (!levelDbFile.exists()) { - logger.info(" {} does not exist.", srcDbPath.toString()); - return false; - } - - DB level = newLevelDb(srcDbPath); - - if (this.dstDbPath.toFile().exists()) { - logger.info(" {} begin to clear exist database directory", this.dbName); - FileUtil.deleteDir(this.dstDbPath.toFile()); - logger.info(" {} clear exist database directory done.", this.dbName); - } - - FileUtil.createDirIfNotExists(dstDir); - RocksDB rocks = newRocksDb(dstDbPath); - - logger.info("Convert database {} start", this.dbName); - boolean result = convertLevelToRocksBatchIterator(level, rocks) - && createEngine(dstDbPath.toString()); - long etime = System.currentTimeMillis(); - - if (result) { - logger.info("Convert database {} successful end with {} key-value {} minutes", - this.dbName, this.srcDbKeyCount, (etime - this.startTime) / 1000.0 / 60); - } else { - logger.info("Convert database {} failure", this.dbName); - if (this.dstDbPath.toFile().exists()) { - logger.info(" {} begin to clear exist database directory", this.dbName); - FileUtil.deleteDir(this.dstDbPath.toFile()); - logger.info(" {} clear exist database directory done.", this.dbName); - } - } - return result; - } - - public long byteArrayToIntWithOne(long sum, byte[] b) { - for (byte oneByte : b) { - sum += oneByte; - } - return sum; - } -} \ No newline at end of file diff --git a/framework/src/main/java/org/tron/program/FullNode.java b/framework/src/main/java/org/tron/program/FullNode.java index bd275de544c..9f2f497a579 100644 --- a/framework/src/main/java/org/tron/program/FullNode.java +++ b/framework/src/main/java/org/tron/program/FullNode.java @@ -21,12 +21,20 @@ public class FullNode { */ public static void main(String[] args) { ExitManager.initExceptionHandler(); - logger.info("Full node running."); Args.setParam(args, Constant.TESTNET_CONF); CommonParameter parameter = Args.getInstance(); LogService.load(parameter.getLogbackPath()); + if (parameter.isSolidityNode()) { + SolidityNode.start(); + return; + } + if (parameter.isKeystoreFactory()) { + KeystoreFactory.start(); + return; + } + logger.info("Full node running."); if (Args.getInstance().isDebug()) { logger.info("in debug mode, it won't check energy time"); } else { diff --git a/framework/src/main/java/org/tron/program/KeystoreFactory.java b/framework/src/main/java/org/tron/program/KeystoreFactory.java index bfd2df22856..8199d7e9076 100755 --- a/framework/src/main/java/org/tron/program/KeystoreFactory.java +++ b/framework/src/main/java/org/tron/program/KeystoreFactory.java @@ -1,6 +1,5 @@ package org.tron.program; -import com.beust.jcommander.JCommander; import java.io.File; import java.io.IOException; import java.util.Scanner; @@ -11,8 +10,6 @@ import org.tron.common.parameter.CommonParameter; import org.tron.common.utils.ByteArray; import org.tron.common.utils.Utils; -import org.tron.core.Constant; -import org.tron.core.config.args.Args; import org.tron.core.exception.CipherException; import org.tron.keystore.Credentials; import org.tron.keystore.WalletUtils; @@ -22,15 +19,8 @@ public class KeystoreFactory { private static final String FilePath = "Wallet"; - public static void main(String[] args) { - Args.setParam(args, Constant.TESTNET_CONF); + public static void start() { KeystoreFactory cli = new KeystoreFactory(); - - JCommander.newBuilder() - .addObject(cli) - .build() - .parse(args); - cli.run(); } diff --git a/framework/src/main/java/org/tron/program/SolidityNode.java b/framework/src/main/java/org/tron/program/SolidityNode.java index b774ab03aaa..3367141e2a5 100644 --- a/framework/src/main/java/org/tron/program/SolidityNode.java +++ b/framework/src/main/java/org/tron/program/SolidityNode.java @@ -5,21 +5,17 @@ import java.util.concurrent.LinkedBlockingDeque; import java.util.concurrent.atomic.AtomicLong; import lombok.extern.slf4j.Slf4j; -import org.apache.commons.lang3.BooleanUtils; import org.springframework.beans.factory.support.DefaultListableBeanFactory; import org.springframework.util.ObjectUtils; import org.tron.common.application.Application; import org.tron.common.application.ApplicationFactory; import org.tron.common.application.TronApplicationContext; import org.tron.common.client.DatabaseGrpcClient; -import org.tron.common.exit.ExitManager; import org.tron.common.parameter.CommonParameter; import org.tron.common.prometheus.Metrics; import org.tron.core.ChainBaseManager; -import org.tron.core.Constant; import org.tron.core.capsule.BlockCapsule; import org.tron.core.config.DefaultConfig; -import org.tron.core.config.args.Args; import org.tron.core.db.Manager; import org.tron.core.exception.TronError; import org.tron.protos.Protocol.Block; @@ -55,25 +51,12 @@ public SolidityNode(Manager dbManager) { /** * Start the SolidityNode. */ - public static void main(String[] args) { - ExitManager.initExceptionHandler(); + public static void start() { logger.info("Solidity node is running."); - Args.setParam(args, Constant.TESTNET_CONF); CommonParameter parameter = CommonParameter.getInstance(); - - logger.info("index switch is {}", - BooleanUtils.toStringOnOff(BooleanUtils - .toBoolean(parameter.getStorage().getIndexSwitch()))); - if (ObjectUtils.isEmpty(parameter.getTrustNodeAddr())) { - logger.error("Trust node is not set."); - return; - } - parameter.setSolidityNode(true); - - if (parameter.isHelp()) { - logger.info("Here is the help message."); - return; + throw new TronError(new IllegalArgumentException("Trust node is not set."), + TronError.ErrCode.SOLID_NODE_INIT); } // init metrics first Metrics.init(); @@ -88,11 +71,11 @@ public static void main(String[] args) { context.registerShutdownHook(); appT.startup(); SolidityNode node = new SolidityNode(appT.getDbManager()); - node.start(); + node.run(); appT.blockUntilShutdown(); } - private void start() { + private void run() { try { new Thread(this::getBlock).start(); new Thread(this::processBlock).start(); diff --git a/framework/src/main/java/org/tron/program/Version.java b/framework/src/main/java/org/tron/program/Version.java index 4e9528ee50e..de3f91f0a5c 100644 --- a/framework/src/main/java/org/tron/program/Version.java +++ b/framework/src/main/java/org/tron/program/Version.java @@ -2,9 +2,9 @@ public class Version { - public static final String VERSION_NAME = "GreatVoyage-v4.7.7-243-gb3555dd655"; - public static final String VERSION_CODE = "18631"; - private static final String VERSION = "4.8.0"; + public static final String VERSION_NAME = "GreatVoyage-v4.8.0.1-1-g44a4bc8263"; + public static final String VERSION_CODE = "18636"; + private static final String VERSION = "4.8.1"; public static String getVersion() { return VERSION; diff --git a/framework/src/main/resources/config-localtest.conf b/framework/src/main/resources/config-localtest.conf index 405a0f92b2d..bdd5ea14d3e 100644 --- a/framework/src/main/resources/config-localtest.conf +++ b/framework/src/main/resources/config-localtest.conf @@ -163,6 +163,7 @@ node { # httpPBFTPort = 8565 # maxBlockRange = 5000 # maxSubTopics = 1000 + # maxBlockFilterNum = 30000 } } diff --git a/framework/src/main/resources/config.conf b/framework/src/main/resources/config.conf index d434d9c7203..54f229e4e25 100644 --- a/framework/src/main/resources/config.conf +++ b/framework/src/main/resources/config.conf @@ -1,90 +1,80 @@ net { + # Type can be 'mainnet' or 'testnet', refers to address type. + # Hex address of 'mainnet' begin with 0x41, and 'testnet' begin with 0xa0. + # Note: 'testnet' is not related to TRON network Nile, Shasta or private net type = mainnet - # type = testnet } storage { # Directory for storing persistent data - db.engine = "LEVELDB", + db.engine = "LEVELDB", // deprecated for arm, because arm only support "ROCKSDB". db.sync = false, db.directory = "database", - index.directory = "index", - transHistory.switch = "on", - # You can custom these 14 databases' configs: - - # account, account-index, asset-issue, block, block-index, - # block_KDB, peers, properties, recent-block, trans, - # utxo, votes, witness, witness_schedule. - # Otherwise, db configs will remain default and data will be stored in - # the path of "output-directory" or which is set by "-d" ("--output-directory"). + # Whether to write transaction result in transactionRetStore + transHistory.switch = "on", - # setting can impove leveldb performance .... start + # setting can improve leveldb performance .... start, deprecated for arm # node: if this will increase process fds,you may be check your ulimit if 'too many open files' error occurs # see https://github.com/tronprotocol/tips/blob/master/tip-343.md for detail - # if you find block sync has lower performance,you can try this settings - #default = { + # if you find block sync has lower performance, you can try this settings + # default = { # maxOpenFiles = 100 - #} - #defaultM = { + # } + # defaultM = { # maxOpenFiles = 500 - #} - #defaultL = { + # } + # defaultL = { # maxOpenFiles = 1000 - #} - # setting can impove leveldb performance .... end + # } + # setting can improve leveldb performance .... end, deprecated for arm - # Attention: name is a required field that must be set !!! + # You can customize the configuration for each database. Otherwise, the database settings will use + # their defaults, and data will be stored in the "output-directory" or in the directory specified + # by the "-d" or "--output-directory" option. Attention: name is a required field that must be set! + # In this configuration, the name and path properties take effect for both LevelDB and RocksDB storage engines, + # while the additional properties (such as createIfMissing, paranoidChecks, compressionType, etc.) only take effect when using LevelDB. properties = [ - // { - // name = "account", - // path = "storage_directory_test", - // createIfMissing = true, - // paranoidChecks = true, - // verifyChecksums = true, - // compressionType = 1, // compressed with snappy - // blockSize = 4096, // 4 KB = 4 * 1024 B - // writeBufferSize = 10485760, // 10 MB = 10 * 1024 * 1024 B - // cacheSize = 10485760, // 10 MB = 10 * 1024 * 1024 B - // maxOpenFiles = 100 - // }, - // { - // name = "account-index", - // path = "storage_directory_test", - // createIfMissing = true, - // paranoidChecks = true, - // verifyChecksums = true, - // compressionType = 1, // compressed with snappy - // blockSize = 4096, // 4 KB = 4 * 1024 B - // writeBufferSize = 10485760, // 10 MB = 10 * 1024 * 1024 B - // cacheSize = 10485760, // 10 MB = 10 * 1024 * 1024 B - // maxOpenFiles = 100 - // }, + # { + # name = "account", + # path = "storage_directory_test", + # createIfMissing = true, // deprecated for arm start + # paranoidChecks = true, + # verifyChecksums = true, + # compressionType = 1, // compressed with snappy + # blockSize = 4096, // 4 KB = 4 * 1024 B + # writeBufferSize = 10485760, // 10 MB = 10 * 1024 * 1024 B + # cacheSize = 10485760, // 10 MB = 10 * 1024 * 1024 B + # maxOpenFiles = 100 // deprecated for arm end + # }, + # { + # name = "account-index", + # path = "storage_directory_test", + # createIfMissing = true, + # paranoidChecks = true, + # verifyChecksums = true, + # compressionType = 1, // compressed with snappy + # blockSize = 4096, // 4 KB = 4 * 1024 B + # writeBufferSize = 10485760, // 10 MB = 10 * 1024 * 1024 B + # cacheSize = 10485760, // 10 MB = 10 * 1024 * 1024 B + # maxOpenFiles = 100 + # }, ] needToUpdateAsset = true - //dbsettings is needed when using rocksdb as the storage implement (db.engine="ROCKSDB"). - //we'd strongly recommend that do not modify it unless you know every item's meaning clearly. + # dbsettings is needed when using rocksdb as the storage implement (db.engine="ROCKSDB"). + # we'd strongly recommend that do not modify it unless you know every item's meaning clearly. dbSettings = { levelNumber = 7 - //compactThreads = 32 + # compactThreads = 32 blocksize = 64 // n * KB maxBytesForLevelBase = 256 // n * MB maxBytesForLevelMultiplier = 10 level0FileNumCompactionTrigger = 4 targetFileSizeBase = 256 // n * MB targetFileSizeMultiplier = 1 - } - - //backup settings when using rocks db as the storage implement (db.engine="ROCKSDB"). - //if you want to use the backup plugin, please confirm set the db.engine="ROCKSDB" above. - backup = { - enable = false // indicate whether enable the backup plugin - propPath = "prop.properties" // record which bak directory is valid - bak1path = "bak1/database" // you must set two backup directories to prevent application halt unexpected(e.g. kill -9). - bak2path = "bak2/database" - frequency = 10000 // indicate backup db once every 10000 blocks processed. + maxOpenFiles = 5000 } balance.history.lookup = false @@ -95,13 +85,16 @@ storage { # the estimated number of block transactions (default 1000, min 100, max 10000). # so the total number of cached transactions is 65536 * txCache.estimatedTransactions # txCache.estimatedTransactions = 1000 - # if true, transaction cache initialization will be faster. default false - # txCache.initOptimization = true - # data root setting, for check data, currently, only reward-vi is used. + # if true, transaction cache initialization will be faster. Default: false + txCache.initOptimization = true + + # The number of blocks flushed to db in each batch during node syncing. Default: 1 + # snapshot.maxFlushCount = 1 + # data root setting, for check data, currently, only reward-vi is used. # merkleRoot = { - # reward-vi = 9debcb9924055500aaae98cdee10501c5c39d4daa75800a996f4bdda73dbccd8 // main-net, Sha256Hash, hexString + # reward-vi = 9debcb9924055500aaae98cdee10501c5c39d4daa75800a996f4bdda73dbccd8 // main-net, Sha256Hash, hexString # } } @@ -119,13 +112,13 @@ node.discovery = { #} node.backup { + # udp listen port, each member should have the same configuration + port = 10001 + # my priority, each member should use different priority priority = 8 - # udp listen port, each member should have the save configuration - port = 10001 - - # time interval to send keepAlive message, each member should have the save configuration + # time interval to send keepAlive message, each member should have the same configuration keepAliveInterval = 3000 # peer's ip list, can't contain mine @@ -135,18 +128,27 @@ node.backup { ] } +# Specify the algorithm for generating a public key from private key. To avoid forks, please do not modify it crypto { engine = "eckey" } -# prometheus metrics start -# node.metrics = { -# prometheus{ -# enable=true -# port="9527" -# } -# } -# prometheus metrics end +node.metrics = { + # prometheus metrics + prometheus { + enable = false + port = 9527 + } + + # influxdb metrics + storageEnable = false # Whether write metrics data into InfluxDb. Default: false. + influxdb { + ip = "" + port = 8086 + database = "" + metricsReportInterval = 10 + } +} node { # trust node for solidity node @@ -161,17 +163,16 @@ node { connection.timeout = 2 fetchBlock.timeout = 200 + # syncFetchBatchNum = 2000 - tcpNettyWorkThreadNum = 0 - - udpNettyWorkThreadNum = 1 + # Number of validate sign thread, default availableProcessors + # validateSignThreadNum = 16 maxConnections = 30 + minConnections = 8 - minActiveConnections = 3 - # Number of validate sign thread, default availableProcessors / 2 - # validateSignThreadNum = 16 + minActiveConnections = 3 maxConnectionsWithSameIp = 2 @@ -179,11 +180,23 @@ node { minParticipationRate = 15 + # allowShieldedTransactionApi = true + + # openPrintLog = true + + # If set to true, SR packs transactions into a block in descending order of fee; otherwise, it packs + # them based on their receive timestamp. Default: false + # openTransactionSort = false + + # The threshold for the number of broadcast transactions received from each peer every second, + # transactions exceeding this threshold will be discarded + # maxTps = 1000 + isOpenFullTcpDisconnect = false inactiveThreshold = 600 //seconds p2p { - version = 11111 # 11111: mainnet; 20180622: testnet + version = 11111 # Mainnet:11111; Nile:201910292; Shasta:1 } active = [ @@ -214,19 +227,6 @@ node { PBFTPort = 8092 } - # use your ipv6 address for node discovery and tcp connection, default false - enableIpv6 = false - - # if your node's highest block num is below than all your pees', try to acquire new connection. default false - effectiveCheckEnable = false - - dns { - # dns urls to get nodes, url format tree://{pubkey}@{domain}, default empty - treeUrls = [ - #"tree://AKMQMNAJJBL73LXWPXDI4I5ZWWIZ4AWO34DWQ636QOBBXNFXH3LQS@main.trondisco.net", - ] - } - rpc { enable = true port = 50051 @@ -234,6 +234,7 @@ node { solidityPort = 50061 PBFTEnable = true PBFTPort = 50071 + # Number of gRPC thread, default availableProcessors / 2 # thread = 16 @@ -255,17 +256,23 @@ node { # The maximum size of header list allowed to be received, default 8192 # maxHeaderListSize = + # The number of RST_STREAM frames allowed to be sent per connection per period for grpc, by default there is no limit. + # maxRstStream = + + # The number of seconds per period for grpc + # secondsPerWindow = + # Transactions can only be broadcast if the number of effective connections is reached. minEffectiveConnection = 1 - # The switch of the reflection service, effective for all gRPC services - # reflectionService = true + # The switch of the reflection service, effective for all gRPC services, used for grpcurl tool. Default: false + reflectionService = false } # number of solidity thread in the FullNode. # If accessing solidity rpc and http interface timeout, could increase the number of threads, # The default value is the number of cpu cores of the machine. - #solidity.threads = 8 + # solidity.threads = 8 # Limits the maximum percentage (default 75%) of producing block interval # to provide sufficient time to perform other operations e.g. broadcast block @@ -274,52 +281,120 @@ node { # Limits the maximum number (default 700) of transaction from network layer # netMaxTrxPerSecond = 700 - # open the history query APIs(http&GRPC) when node is a lite fullNode, - # like {getBlockByNum, getBlockByID, getTransactionByID...}. - # default: false. + # Whether to enable the node detection function. Default: false + # nodeDetectEnable = false + + # use your ipv6 address for node discovery and tcp connection. Default: false + # enableIpv6 = false + + # if your node's highest block num is below than all your pees', try to acquire new connection. Default: false + # effectiveCheckEnable = false + + # Dynamic loading configuration function, disabled by default + dynamicConfig = { + # enable = false + # checkInterval = 600 // Check interval of Configuration file's change, default is 600 seconds + } + + # Whether to continue broadcast transactions after at least maxUnsolidifiedBlocks are not solidified. Default: false + # unsolidifiedBlockCheck = false + # maxUnsolidifiedBlocks = 54 + + dns { + # dns urls to get nodes, url format tree://{pubkey}@{domain}, default empty + treeUrls = [ + #"tree://AKMQMNAJJBL73LXWPXDI4I5ZWWIZ4AWO34DWQ636QOBBXNFXH3LQS@main.trondisco.net", + ] + + # enable or disable dns publish. Default: false + # publish = false + + # dns domain to publish nodes, required if publish is true + # dnsDomain = "nodes1.example.org" + + # dns private key used to publish, required if publish is true, hex string of length 64 + # dnsPrivate = "b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291" + + # known dns urls to publish if publish is true, url format tree://{pubkey}@{domain}, default empty + # knownUrls = [ + #"tree://APFGGTFOBVE2ZNAB3CSMNNX6RRK3ODIRLP2AA5U4YFAA6MSYZUYTQ@nodes2.example.org", + # ] + + # staticNodes = [ + # static nodes to published on dns + # Sample entries: + # "ip:port", + # "ip:port" + # ] + + # merge several nodes into a leaf of tree, should be 1~5 + # maxMergeSize = 5 + + # only nodes change percent is bigger then the threshold, we update data on dns + # changeThreshold = 0.1 + + # dns server to publish, required if publish is true, only aws or aliyun is support + # serverType = "aws" + + # access key id of aws or aliyun api, required if publish is true, string + # accessKeyId = "your-key-id" + + # access key secret of aws or aliyun api, required if publish is true, string + # accessKeySecret = "your-key-secret" + + # if publish is true and serverType is aliyun, it's endpoint of aws dns server, string + # aliyunDnsEndpoint = "alidns.aliyuncs.com" + + # if publish is true and serverType is aws, it's region of aws api, such as "eu-south-1", string + # awsRegion = "us-east-1" + + # if publish is true and server-type is aws, it's host zone id of aws's domain, string + # awsHostZoneId = "your-host-zone-id" + } + + # open the history query APIs(http&GRPC) when node is a lite FullNode, + # like {getBlockByNum, getBlockByID, getTransactionByID...}. Default: false. # note: above APIs may return null even if blocks and transactions actually are on the blockchain - # when opening on a lite fullnode. only open it if the consequences being clearly known + # when opening on a lite FullNode. only open it if the consequences being clearly known # openHistoryQueryWhenLiteFN = false jsonrpc { - # Note: If you turn on jsonrpc and run it for a while and then turn it off, you will not - # be able to get the data from eth_getLogs for that period of time. - - # httpFullNodeEnable = true + # Note: Before release_4.8.1, if you turn on jsonrpc and run it for a while and then turn it off, + # you will not be able to get the data from eth_getLogs for that period of time. Default: false + # httpFullNodeEnable = false # httpFullNodePort = 8545 - # httpSolidityEnable = true + # httpSolidityEnable = false # httpSolidityPort = 8555 - # httpPBFTEnable = true + # httpPBFTEnable = false # httpPBFTPort = 8565 # The maximum blocks range to retrieve logs for eth_getLogs, default value is 5000, # should be > 0, otherwise means no limit. - # maxBlockRange = 5000 + maxBlockRange = 5000 # The maximum number of allowed topics within a topic criteria, default value is 1000, # should be > 0, otherwise means no limit. - # maxSubTopics = 1000 + maxSubTopics = 1000 + # Allowed maximum number for blockFilter + maxBlockFilterNum = 50000 } - # Disabled api list, it will work for http, rpc and pbft, both fullnode and soliditynode, - # but not jsonrpc. - # Sample: The setting is case insensitive, GetNowBlock2 is equal to getnowblock2 - # - # disabledApi = [ - # "getaccount", - # "getnowblock2" - # ] + # Disabled api list, it will work for http, rpc and pbft, both FullNode and SolidityNode, + # but not jsonrpc. The setting is case insensitive, GetNowBlock2 is equal to getnowblock2 + disabledApi = [ + # "getaccount", + # "getnowblock2" + ] } ## rate limiter config rate.limiter = { - # Every api could be set a specific rate limit strategy. Three strategy are supported:GlobalPreemptibleAdapter、IPQPSRateLimiterAdapte、QpsRateLimiterAdapter - # GlobalPreemptibleAdapter: permit is the number of preemptible resource, every client must apply one resourse - # before do the request and release the resource after got the reponse automaticlly. permit should be a Integer. + # Every api could only set a specific rate limit strategy. Three blocking strategy are supported: + # GlobalPreemptibleAdapter: The number of preemptible resource or maximum concurrent requests globally. # QpsRateLimiterAdapter: qps is the average request count in one second supported by the server, it could be a Double or a Integer. # IPQPSRateLimiterAdapter: similar to the QpsRateLimiterAdapter, qps could be a Double or a Integer. - # If do not set, the "default strategy" is set.The "default startegy" is based on QpsRateLimiterAdapter, the qps is set as 10000. + # If not set, QpsRateLimiterAdapter with qps=1000 is the default strategy. # # Sample entries: # @@ -363,9 +438,20 @@ rate.limiter = { # }, ] + p2p = { + # syncBlockChain = 3.0 + # fetchInvData = 3.0 + # disconnect = 1.0 + } + + # global qps, default 50000 + global.qps = 50000 + # IP-based global qps, default 10000 + global.ip.qps = 10000 } + seed.node = { # List of the seed nodes # Seed nodes are stable full nodes @@ -375,18 +461,18 @@ seed.node = { # "ip:port" # ] ip.list = [ - "54.236.37.243:18888", - "52.53.189.99:18888", - "18.196.99.16:18888", - "34.253.187.192:18888", - "52.56.56.149:18888", - "35.180.51.163:18888", - "54.252.224.209:18888", - "18.228.15.36:18888", - "52.15.93.92:18888", - "34.220.77.106:18888", - "13.127.47.162:18888", - "13.124.62.58:18888", + "3.225.171.164:18888", + "52.8.46.215:18888", + "3.79.71.167:18888", + "108.128.110.16:18888", + "18.133.82.227:18888", + "35.180.81.133:18888", + "13.210.151.5:18888", + "18.231.27.82:18888", + "3.12.212.122:18888", + "52.24.128.7:18888", + "15.207.144.3:18888", + "3.39.38.55:18888", "54.151.226.240:18888", "35.174.93.198:18888", "18.210.241.149:18888", @@ -404,7 +490,9 @@ seed.node = { "54.82.161.39:18888", "54.179.207.68:18888", "18.142.82.44:18888", - "18.163.230.203:18888" + "18.163.230.203:18888", + # "[2a05:d014:1f2f:2600:1b15:921:d60b:4c60]:18888", // use this if support ipv6 + # "[2600:1f18:7260:f400:8947:ebf3:78a0:282b]:18888", // use this if support ipv6 ] } @@ -569,34 +657,33 @@ genesis.block = { } ] - timestamp = "0" #2017-8-26 12:00:00 + timestamp = "0" # Genesis block timestamp, milli seconds parentHash = "0xe58f33f9baf9305dc6f82b9f1934ea8f0ade2defb951258d50167028c780351f" } -// Optional.The default is empty. -// It is used when the witness account has set the witnessPermission. -// When it is not empty, the localWitnessAccountAddress represents the address of the witness account, -// and the localwitness is configured with the private key of the witnessPermissionAddress in the witness account. -// When it is empty,the localwitness is configured with the private key of the witness account. - -//localWitnessAccountAddress = +# Optional. The default is empty. It is used when the witness account has set the witnessPermission. +# When it is not empty, the localWitnessAccountAddress represents the address of the witness account, +# and the localwitness is configured with the private key of the witnessPermissionAddress in the witness account. +# When it is empty,the localwitness is configured with the private key of the witness account. +# localWitnessAccountAddress = localwitness = [ ] -#localwitnesskeystore = [ +# localwitnesskeystore = [ # "localwitnesskeystore.json" -#] +# ] block = { needSyncCheck = true - maintenanceTimeInterval = 21600000 - proposalExpireTime = 259200000 // 3 day: 259200000(ms) + maintenanceTimeInterval = 21600000 // 6 hours: 21600000(ms) + proposalExpireTime = 259200000 // default value: 3 days: 259200000(ms), Note: this value is controlled by committee proposal + # checkFrozenTime = 1 // for test only } -# Transaction reference block, default is "solid", configure to "head" may accur TaPos error -# trx.reference.block = "solid" // head;solid; +# Transaction reference block, default is "solid", configure to "head" may cause TaPos error +trx.reference.block = "solid" // "head" or "solid" # This property sets the number of milliseconds after the creation of the transaction that is expired, default value is 60000. # trx.expiration.timeInMilliseconds = 60000 @@ -607,26 +694,73 @@ vm = { minTimeRatio = 0.0 maxTimeRatio = 5.0 saveInternalTx = false + # lruCacheSize = 500 + # vmTrace = false - # Indicates whether the node stores featured internal transactions, such as freeze, vote and so on + # Indicates whether the node stores featured internal transactions, such as freeze, vote and so on. Default: false. # saveFeaturedInternalTx = false - # Indicates whether the node stores the details of the internal transactions generated by the CANCELALLUNFREEZEV2 opcode, such as bandwidth/energy/tronpower cancel amount. + # Indicates whether the node stores the details of the internal transactions generated by the CANCELALLUNFREEZEV2 opcode, + # such as bandwidth/energy/tronpower cancel amount. Default: false. # saveCancelAllUnfreezeV2Details = false # In rare cases, transactions that will be within the specified maximum execution time (default 10(ms)) are re-executed and packaged # longRunningTime = 10 - # Indicates whether the node support estimate energy API. + # Indicates whether the node support estimate energy API. Default: false. # estimateEnergy = false - # Indicates the max retry time for executing transaction in estimating energy. + # Indicates the max retry time for executing transaction in estimating energy. Default 3. # estimateEnergyMaxRetry = 3 } +# These parameters are designed for private chain testing only and cannot be freely switched on or off in production systems. committee = { allowCreationOfContracts = 0 //mainnet:0 (reset by committee),test:1 allowAdaptiveEnergy = 0 //mainnet:0 (reset by committee),test:1 + # allowCreationOfContracts = 0 + # allowMultiSign = 0 + # allowAdaptiveEnergy = 0 + # allowDelegateResource = 0 + # allowSameTokenName = 0 + # allowTvmTransferTrc10 = 0 + # allowTvmConstantinople = 0 + # allowTvmSolidity059 = 0 + # forbidTransferToContract = 0 + # allowShieldedTRC20Transaction = 0 + # allowTvmIstanbul = 0 + # allowMarketTransaction = 0 + # allowProtoFilterNum = 0 + # allowAccountStateRoot = 0 + # changedDelegation = 0 + # allowPBFT = 0 + # pBFTExpireNum = 0 + # allowTransactionFeePool = 0 + # allowBlackHoleOptimization = 0 + # allowNewResourceModel = 0 + # allowReceiptsMerkleRoot = 0 + # allowTvmFreeze = 0 + # allowTvmVote = 0 + # unfreezeDelayDays = 0 + # allowTvmLondon = 0 + # allowTvmCompatibleEvm = 0 + # allowNewRewardAlgorithm = 0 + # allowAccountAssetOptimization = 0 + # allowAssetOptimization = 0 + # allowNewReward = 0 + # memoFee = 0 + # allowDelegateOptimization = 0 + # allowDynamicEnergy = 0 + # dynamicEnergyThreshold = 0 + # dynamicEnergyMaxFactor = 0 + # allowTvmShangHai = 0 + # allowOldRewardOpt = 0 + # allowEnergyAdjustment = 0 + # allowStrictMath = 0 + # allowTvmCancun = 0 + # allowTvmBlob = 0 + # consensusLogicOptimization = 0 + # allowOptimizedReturnValueOfChainId = 0 } event.subscribe = { @@ -635,24 +769,35 @@ event.subscribe = { bindport = 5555 // bind port sendqueuelength = 1000 //max length of send queue } + version = 0 + # Specify the starting block number to sync historical events. This is only applicable when version = 1. + # After performing a full event sync, set this value to 0 or a negative number. + # startSyncBlockNum = 1 path = "" // absolute path of plugin server = "" // target server address to receive event triggers - dbconfig = "" // dbname|username|password - contractParse = true, + # dbname|username|password, if you want to create indexes for collections when the collections + # are not exist, you can add version and set it to 2, as dbname|username|password|version + # if you use version 2 and one collection not exists, it will create index automaticaly; + # if you use version 2 and one collection exists, it will not create index, you must create index manually; + dbconfig = "" + contractParse = true topics = [ { triggerName = "block" // block trigger, the value can't be modified enable = false topic = "block" // plugin topic, the value could be modified + solidified = false // if set true, just need solidified block. Default: false }, { triggerName = "transaction" enable = false topic = "transaction" + solidified = false + ethCompatible = false // if set true, add transactionIndex, cumulativeEnergyUsed, preCumulativeLogCount, logList, energyUnitPrice. Default: false }, { - triggerName = "contractevent" + triggerName = "contractevent" // contractevent represents contractlog data decoded by the ABI. enable = false topic = "contractevent" }, @@ -660,10 +805,11 @@ event.subscribe = { triggerName = "contractlog" enable = false topic = "contractlog" + redundancy = false // if set true, contractevent will also be regarded as contractlog }, { - triggerName = "solidity" // solidity block event trigger, the value can't be modified - enable = true // the default value is true + triggerName = "solidity" // solidity block trigger(just include solidity block number and timestamp), the value can't be modified + enable = true // Default: true topic = "solidity" }, { @@ -675,6 +821,7 @@ event.subscribe = { triggerName = "soliditylog" enable = false topic = "soliditylog" + redundancy = false // if set true, solidityevent will also be regarded as soliditylog } ] @@ -689,5 +836,4 @@ event.subscribe = { "" // contract topic you want to subscribe, if it's set to "", you will receive contract logs/events with any contract topic. ] } - } diff --git a/framework/src/test/java/org/springframework/http/InvalidMediaTypeException.java b/framework/src/test/java/org/springframework/http/InvalidMediaTypeException.java new file mode 100644 index 00000000000..3b1e41f1756 --- /dev/null +++ b/framework/src/test/java/org/springframework/http/InvalidMediaTypeException.java @@ -0,0 +1,61 @@ +/* + * Copyright 2002-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.http; + +import org.springframework.util.InvalidMimeTypeException; + +/** + * Exception thrown from {@link MediaType#parseMediaType(String)} in case of + * encountering an invalid media type specification String. + * + * @author Juergen Hoeller + * @since 3.2.2 + */ +@SuppressWarnings("serial") +public class InvalidMediaTypeException extends IllegalArgumentException { + + private final String mediaType; + + + /** + * Create a new InvalidMediaTypeException for the given media type. + * + * @param mediaType the offending media type + * @param message a detail message indicating the invalid part + */ + public InvalidMediaTypeException(String mediaType, String message) { + super("Invalid media type \"" + mediaType + "\": " + message); + this.mediaType = mediaType; + } + + /** + * Constructor that allows wrapping {@link InvalidMimeTypeException}. + */ + InvalidMediaTypeException(InvalidMimeTypeException ex) { + super(ex.getMessage(), ex); + this.mediaType = ex.getMimeType(); + } + + + /** + * Return the offending media type. + */ + public String getMediaType() { + return this.mediaType; + } + +} \ No newline at end of file diff --git a/framework/src/test/java/org/springframework/http/MediaType.java b/framework/src/test/java/org/springframework/http/MediaType.java new file mode 100644 index 00000000000..84962970235 --- /dev/null +++ b/framework/src/test/java/org/springframework/http/MediaType.java @@ -0,0 +1,841 @@ +/* + * Copyright 2002-2022 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.http; + +import java.io.Serializable; +import java.nio.charset.Charset; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.Comparator; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import org.springframework.lang.Nullable; +import org.springframework.util.Assert; +import org.springframework.util.CollectionUtils; +import org.springframework.util.InvalidMimeTypeException; +import org.springframework.util.MimeType; +import org.springframework.util.MimeTypeUtils; +import org.springframework.util.StringUtils; + +/** + * A subclass of {@link MimeType} that adds support for quality parameters + * as defined in the HTTP specification. + * + * @author Arjen Poutsma + * @author Juergen Hoeller + * @author Rossen Stoyanchev + * @author Sebastien Deleuze + * @author Kazuki Shimizu + * @author Sam Brannen + * @see + * HTTP 1.1: Semantics and Content, section 3.1.1.1 + * @since 3.0 + */ +public class MediaType extends MimeType implements Serializable { + + /** + * Public constant media type that includes all media ranges (i.e. "*/*"). + */ + public static final MediaType ALL; + /** + * A String equivalent of {@link MediaType#ALL}. + */ + public static final String ALL_VALUE = "*/*"; + /** + * Public constant media type for {@code application/atom+xml}. + */ + public static final MediaType APPLICATION_ATOM_XML; + /** + * A String equivalent of {@link MediaType#APPLICATION_ATOM_XML}. + */ + public static final String APPLICATION_ATOM_XML_VALUE = "application/atom+xml"; + /** + * Public constant media type for {@code application/cbor}. + * + * @since 5.2 + */ + public static final MediaType APPLICATION_CBOR; + /** + * A String equivalent of {@link MediaType#APPLICATION_CBOR}. + * + * @since 5.2 + */ + public static final String APPLICATION_CBOR_VALUE = "application/cbor"; + /** + * Public constant media type for {@code application/x-www-form-urlencoded}. + */ + public static final MediaType APPLICATION_FORM_URLENCODED; + /** + * A String equivalent of {@link MediaType#APPLICATION_FORM_URLENCODED}. + */ + public static final String APPLICATION_FORM_URLENCODED_VALUE = + "application/x-www-form-urlencoded"; + /** + * Public constant media type for {@code application/graphql+json}. + * + * @see GraphQL over HTTP spec + * @since 5.3.19 + */ + public static final MediaType APPLICATION_GRAPHQL; + /** + * A String equivalent of {@link MediaType#APPLICATION_GRAPHQL}. + * + * @since 5.3.19 + */ + public static final String APPLICATION_GRAPHQL_VALUE = "application/graphql+json"; + /** + * Public constant media type for {@code application/json}. + */ + public static final MediaType APPLICATION_JSON; + /** + * A String equivalent of {@link MediaType#APPLICATION_JSON}. + * + * @see #APPLICATION_JSON_UTF8_VALUE + */ + public static final String APPLICATION_JSON_VALUE = "application/json"; + /** + * Public constant media type for {@code application/json;charset=UTF-8}. + * + * @deprecated as of 5.2 in favor of {@link #APPLICATION_JSON} + * since major browsers like Chrome + * + * now comply with the specification and interpret correctly UTF-8 special + * characters without requiring a {@code charset=UTF-8} parameter. + */ + @Deprecated + public static final MediaType APPLICATION_JSON_UTF8; + /** + * A String equivalent of {@link MediaType#APPLICATION_JSON_UTF8}. + * + * @deprecated as of 5.2 in favor of {@link #APPLICATION_JSON_VALUE} + * since major browsers like Chrome + * + * now comply with the specification and interpret correctly UTF-8 special + * characters without requiring a {@code charset=UTF-8} parameter. + */ + @Deprecated + public static final String APPLICATION_JSON_UTF8_VALUE = "application/json;charset=UTF-8"; + /** + * Public constant media type for {@code application/octet-stream}. + */ + public static final MediaType APPLICATION_OCTET_STREAM; + /** + * A String equivalent of {@link MediaType#APPLICATION_OCTET_STREAM}. + */ + public static final String APPLICATION_OCTET_STREAM_VALUE = "application/octet-stream"; + /** + * Public constant media type for {@code application/pdf}. + * + * @since 4.3 + */ + public static final MediaType APPLICATION_PDF; + /** + * A String equivalent of {@link MediaType#APPLICATION_PDF}. + * + * @since 4.3 + */ + public static final String APPLICATION_PDF_VALUE = "application/pdf"; + /** + * Public constant media type for {@code application/problem+json}. + * + * @see + * Problem Details for HTTP APIs, 6.1. application/problem+json + * @since 5.0 + */ + public static final MediaType APPLICATION_PROBLEM_JSON; + /** + * A String equivalent of {@link MediaType#APPLICATION_PROBLEM_JSON}. + * + * @since 5.0 + */ + public static final String APPLICATION_PROBLEM_JSON_VALUE = "application/problem+json"; + /** + * Public constant media type for {@code application/problem+json}. + * + * @see + * Problem Details for HTTP APIs, 6.1. application/problem+json + * @since 5.0 + * @deprecated as of 5.2 in favor of {@link #APPLICATION_PROBLEM_JSON} + * since major browsers like Chrome + * + * now comply with the specification and interpret correctly UTF-8 special + * characters without requiring a {@code charset=UTF-8} parameter. + */ + @Deprecated + public static final MediaType APPLICATION_PROBLEM_JSON_UTF8; + /** + * A String equivalent of {@link MediaType#APPLICATION_PROBLEM_JSON_UTF8}. + * + * @since 5.0 + * @deprecated as of 5.2 in favor of {@link #APPLICATION_PROBLEM_JSON_VALUE} + * since major browsers like Chrome + * + * now comply with the specification and interpret correctly UTF-8 special + * characters without requiring a {@code charset=UTF-8} parameter. + */ + @Deprecated + public static final String APPLICATION_PROBLEM_JSON_UTF8_VALUE = + "application/problem+json;charset=UTF-8"; + /** + * Public constant media type for {@code application/problem+xml}. + * + * @see + * Problem Details for HTTP APIs, 6.2. application/problem+xml + * @since 5.0 + */ + public static final MediaType APPLICATION_PROBLEM_XML; + /** + * A String equivalent of {@link MediaType#APPLICATION_PROBLEM_XML}. + * + * @since 5.0 + */ + public static final String APPLICATION_PROBLEM_XML_VALUE = "application/problem+xml"; + /** + * Public constant media type for {@code application/rss+xml}. + * + * @since 4.3.6 + */ + public static final MediaType APPLICATION_RSS_XML; + /** + * A String equivalent of {@link MediaType#APPLICATION_RSS_XML}. + * + * @since 4.3.6 + */ + public static final String APPLICATION_RSS_XML_VALUE = "application/rss+xml"; + /** + * Public constant media type for {@code application/x-ndjson}. + * + * @since 5.3 + */ + public static final MediaType APPLICATION_NDJSON; + /** + * A String equivalent of {@link MediaType#APPLICATION_NDJSON}. + * + * @since 5.3 + */ + public static final String APPLICATION_NDJSON_VALUE = "application/x-ndjson"; + /** + * Public constant media type for {@code application/stream+json}. + * + * @since 5.0 + * @deprecated as of 5.3, see notice on {@link #APPLICATION_STREAM_JSON_VALUE}. + */ + @Deprecated + public static final MediaType APPLICATION_STREAM_JSON; + /** + * A String equivalent of {@link MediaType#APPLICATION_STREAM_JSON}. + * + * @since 5.0 + * @deprecated as of 5.3 since it originates from the W3C Activity Streams + * specification which has a more specific purpose and has been since + * replaced with a different mime type. Use {@link #APPLICATION_NDJSON} as + * a replacement or any other line-delimited JSON format (e.g. JSON Lines, + * JSON Text Sequences). + */ + @Deprecated + public static final String APPLICATION_STREAM_JSON_VALUE = "application/stream+json"; + /** + * Public constant media type for {@code application/xhtml+xml}. + */ + public static final MediaType APPLICATION_XHTML_XML; + /** + * A String equivalent of {@link MediaType#APPLICATION_XHTML_XML}. + */ + public static final String APPLICATION_XHTML_XML_VALUE = "application/xhtml+xml"; + /** + * Public constant media type for {@code application/xml}. + */ + public static final MediaType APPLICATION_XML; + /** + * A String equivalent of {@link MediaType#APPLICATION_XML}. + */ + public static final String APPLICATION_XML_VALUE = "application/xml"; + /** + * Public constant media type for {@code image/gif}. + */ + public static final MediaType IMAGE_GIF; + /** + * A String equivalent of {@link MediaType#IMAGE_GIF}. + */ + public static final String IMAGE_GIF_VALUE = "image/gif"; + /** + * Public constant media type for {@code image/jpeg}. + */ + public static final MediaType IMAGE_JPEG; + /** + * A String equivalent of {@link MediaType#IMAGE_JPEG}. + */ + public static final String IMAGE_JPEG_VALUE = "image/jpeg"; + /** + * Public constant media type for {@code image/png}. + */ + public static final MediaType IMAGE_PNG; + /** + * A String equivalent of {@link MediaType#IMAGE_PNG}. + */ + public static final String IMAGE_PNG_VALUE = "image/png"; + /** + * Public constant media type for {@code multipart/form-data}. + */ + public static final MediaType MULTIPART_FORM_DATA; + /** + * A String equivalent of {@link MediaType#MULTIPART_FORM_DATA}. + */ + public static final String MULTIPART_FORM_DATA_VALUE = "multipart/form-data"; + /** + * Public constant media type for {@code multipart/mixed}. + * + * @since 5.2 + */ + public static final MediaType MULTIPART_MIXED; + /** + * A String equivalent of {@link MediaType#MULTIPART_MIXED}. + * + * @since 5.2 + */ + public static final String MULTIPART_MIXED_VALUE = "multipart/mixed"; + /** + * Public constant media type for {@code multipart/related}. + * + * @since 5.2.5 + */ + public static final MediaType MULTIPART_RELATED; + /** + * A String equivalent of {@link MediaType#MULTIPART_RELATED}. + * + * @since 5.2.5 + */ + public static final String MULTIPART_RELATED_VALUE = "multipart/related"; + /** + * Public constant media type for {@code text/event-stream}. + * + * @see Server-Sent Events W3C recommendation + * @since 4.3.6 + */ + public static final MediaType TEXT_EVENT_STREAM; + /** + * A String equivalent of {@link MediaType#TEXT_EVENT_STREAM}. + * + * @since 4.3.6 + */ + public static final String TEXT_EVENT_STREAM_VALUE = "text/event-stream"; + /** + * Public constant media type for {@code text/html}. + */ + public static final MediaType TEXT_HTML; + /** + * A String equivalent of {@link MediaType#TEXT_HTML}. + */ + public static final String TEXT_HTML_VALUE = "text/html"; + /** + * Public constant media type for {@code text/markdown}. + * + * @since 4.3 + */ + public static final MediaType TEXT_MARKDOWN; + /** + * A String equivalent of {@link MediaType#TEXT_MARKDOWN}. + * + * @since 4.3 + */ + public static final String TEXT_MARKDOWN_VALUE = "text/markdown"; + /** + * Public constant media type for {@code text/plain}. + */ + public static final MediaType TEXT_PLAIN; + /** + * A String equivalent of {@link MediaType#TEXT_PLAIN}. + */ + public static final String TEXT_PLAIN_VALUE = "text/plain"; + /** + * Public constant media type for {@code text/xml}. + */ + public static final MediaType TEXT_XML; + /** + * A String equivalent of {@link MediaType#TEXT_XML}. + */ + public static final String TEXT_XML_VALUE = "text/xml"; + private static final long serialVersionUID = 2069937152339670231L; + private static final String PARAM_QUALITY_FACTOR = "q"; + /** + * Comparator used by {@link #sortByQualityValue(List)}. + */ + public static final Comparator QUALITY_VALUE_COMPARATOR = (mediaType1, mediaType2) -> { + double quality1 = mediaType1.getQualityValue(); + double quality2 = mediaType2.getQualityValue(); + int qualityComparison = Double.compare(quality2, quality1); + if (qualityComparison != 0) { + return qualityComparison; // audio/*;q=0.7 < audio/*;q=0.3 + } else if (mediaType1.isWildcardType() && !mediaType2.isWildcardType()) { // */* < audio/* + return 1; + } else if (mediaType2.isWildcardType() && !mediaType1.isWildcardType()) { // audio/* > */* + return -1; + } else if (!mediaType1.getType().equals(mediaType2.getType())) { // audio/basic == text/html + return 0; + } else { // mediaType1.getType().equals(mediaType2.getType()) + if (mediaType1.isWildcardSubtype() && !mediaType2.isWildcardSubtype()) { + // audio/* < audio/basic + return 1; + } else if (mediaType2.isWildcardSubtype() && !mediaType1.isWildcardSubtype()) { + // audio/basic > audio/* + return -1; + } else if (!mediaType1.getSubtype().equals(mediaType2.getSubtype())) { + // audio/basic == audio/wave + return 0; + } else { + int paramsSize1 = mediaType1.getParameters().size(); + int paramsSize2 = mediaType2.getParameters().size(); + return Integer.compare(paramsSize2, paramsSize1); // audio/basic;level=1 < audio/basic + } + } + }; + /** + * Comparator used by {@link #sortBySpecificity(List)}. + */ + public static final Comparator SPECIFICITY_COMPARATOR = + new SpecificityComparator() { + + @Override + protected int compareParameters(MediaType mediaType1, MediaType mediaType2) { + double quality1 = mediaType1.getQualityValue(); + double quality2 = mediaType2.getQualityValue(); + int qualityComparison = Double.compare(quality2, quality1); + if (qualityComparison != 0) { + return qualityComparison; // audio/*;q=0.7 < audio/*;q=0.3 + } + return super.compareParameters(mediaType1, mediaType2); + } + }; + + static { + // Not using "valueOf' to avoid static init cost + ALL = new MediaType("*", "*"); + APPLICATION_ATOM_XML = new MediaType("application", "atom+xml"); + APPLICATION_CBOR = new MediaType("application", "cbor"); + APPLICATION_FORM_URLENCODED = new MediaType("application", "x-www-form-urlencoded"); + APPLICATION_GRAPHQL = new MediaType("application", "graphql+json"); + APPLICATION_JSON = new MediaType("application", "json"); + APPLICATION_JSON_UTF8 = new MediaType("application", "json", StandardCharsets.UTF_8); + APPLICATION_NDJSON = new MediaType("application", "x-ndjson"); + APPLICATION_OCTET_STREAM = new MediaType("application", "octet-stream"); + APPLICATION_PDF = new MediaType("application", "pdf"); + APPLICATION_PROBLEM_JSON = new MediaType("application", "problem+json"); + APPLICATION_PROBLEM_JSON_UTF8 = new MediaType("application", "problem+json", + StandardCharsets.UTF_8); + APPLICATION_PROBLEM_XML = new MediaType("application", "problem+xml"); + APPLICATION_RSS_XML = new MediaType("application", "rss+xml"); + APPLICATION_STREAM_JSON = new MediaType("application", "stream+json"); + APPLICATION_XHTML_XML = new MediaType("application", "xhtml+xml"); + APPLICATION_XML = new MediaType("application", "xml"); + IMAGE_GIF = new MediaType("image", "gif"); + IMAGE_JPEG = new MediaType("image", "jpeg"); + IMAGE_PNG = new MediaType("image", "png"); + MULTIPART_FORM_DATA = new MediaType("multipart", "form-data"); + MULTIPART_MIXED = new MediaType("multipart", "mixed"); + MULTIPART_RELATED = new MediaType("multipart", "related"); + TEXT_EVENT_STREAM = new MediaType("text", "event-stream"); + TEXT_HTML = new MediaType("text", "html"); + TEXT_MARKDOWN = new MediaType("text", "markdown"); + TEXT_PLAIN = new MediaType("text", "plain"); + TEXT_XML = new MediaType("text", "xml"); + } + + /** + * Create a new {@code MediaType} for the given primary type. + *

The {@linkplain #getSubtype() subtype} is set to "*", parameters empty. + * + * @param type the primary type + * @throws IllegalArgumentException if any of the parameters contain illegal characters + */ + public MediaType(String type) { + super(type); + } + + /** + * Create a new {@code MediaType} for the given primary type and subtype. + *

The parameters are empty. + * + * @param type the primary type + * @param subtype the subtype + * @throws IllegalArgumentException if any of the parameters contain illegal characters + */ + public MediaType(String type, String subtype) { + super(type, subtype, Collections.emptyMap()); + } + + /** + * Create a new {@code MediaType} for the given type, subtype, and character set. + * + * @param type the primary type + * @param subtype the subtype + * @param charset the character set + * @throws IllegalArgumentException if any of the parameters contain illegal characters + */ + public MediaType(String type, String subtype, Charset charset) { + super(type, subtype, charset); + } + + /** + * Create a new {@code MediaType} for the given type, subtype, and quality value. + * + * @param type the primary type + * @param subtype the subtype + * @param qualityValue the quality value + * @throws IllegalArgumentException if any of the parameters contain illegal characters + */ + public MediaType(String type, String subtype, double qualityValue) { + this(type, subtype, Collections.singletonMap(PARAM_QUALITY_FACTOR, + Double.toString(qualityValue))); + } + + /** + * Copy-constructor that copies the type, subtype and parameters of the given + * {@code MediaType}, and allows to set the specified character set. + * + * @param other the other media type + * @param charset the character set + * @throws IllegalArgumentException if any of the parameters contain illegal characters + * @since 4.3 + */ + public MediaType(MediaType other, Charset charset) { + super(other, charset); + } + + /** + * Copy-constructor that copies the type and subtype of the given {@code MediaType}, + * and allows for different parameters. + * + * @param other the other media type + * @param parameters the parameters, may be {@code null} + * @throws IllegalArgumentException if any of the parameters contain illegal characters + */ + public MediaType(MediaType other, @Nullable Map parameters) { + super(other.getType(), other.getSubtype(), parameters); + } + + + /** + * Create a new {@code MediaType} for the given type, subtype, and parameters. + * + * @param type the primary type + * @param subtype the subtype + * @param parameters the parameters, may be {@code null} + * @throws IllegalArgumentException if any of the parameters contain illegal characters + */ + public MediaType(String type, String subtype, @Nullable Map parameters) { + super(type, subtype, parameters); + } + + /** + * Create a new {@code MediaType} for the given {@link MimeType}. + * The type, subtype and parameters information is copied and {@code MediaType}-specific + * checks on parameters are performed. + * + * @param mimeType the MIME type + * @throws IllegalArgumentException if any of the parameters contain illegal characters + * @since 5.3 + */ + public MediaType(MimeType mimeType) { + super(mimeType); + getParameters().forEach(this::checkParameters); + } + + /** + * Parse the given String value into a {@code MediaType} object, + * with this method name following the 'valueOf' naming convention + * (as supported by {@link org.springframework.core.convert.ConversionService}. + * + * @param value the string to parse + * @throws InvalidMediaTypeException if the media type value cannot be parsed + * @see #parseMediaType(String) + */ + public static MediaType valueOf(String value) { + return parseMediaType(value); + } + + /** + * Parse the given String into a single {@code MediaType}. + * + * @param mediaType the string to parse + * @return the media type + * @throws InvalidMediaTypeException if the media type value cannot be parsed + */ + public static MediaType parseMediaType(String mediaType) { + MimeType type; + try { + type = MimeTypeUtils.parseMimeType(mediaType); + } catch (InvalidMimeTypeException ex) { + throw new InvalidMediaTypeException(ex); + } + try { + return new MediaType(type); + } catch (IllegalArgumentException ex) { + throw new InvalidMediaTypeException(mediaType, ex.getMessage()); + } + } + + /** + * Parse the comma-separated string into a list of {@code MediaType} objects. + *

This method can be used to parse an Accept or Content-Type header. + * + * @param mediaTypes the string to parse + * @return the list of media types + * @throws InvalidMediaTypeException if the media type value cannot be parsed + */ + public static List parseMediaTypes(@Nullable String mediaTypes) { + if (!StringUtils.hasLength(mediaTypes)) { + return Collections.emptyList(); + } + // Avoid using java.util.stream.Stream in hot paths + List tokenizedTypes = MimeTypeUtils.tokenize(mediaTypes); + List result = new ArrayList<>(tokenizedTypes.size()); + for (String type : tokenizedTypes) { + if (StringUtils.hasText(type)) { + result.add(parseMediaType(type)); + } + } + return result; + } + + /** + * Parse the given list of (potentially) comma-separated strings into a + * list of {@code MediaType} objects. + *

This method can be used to parse an Accept or Content-Type header. + * + * @param mediaTypes the string to parse + * @return the list of media types + * @throws InvalidMediaTypeException if the media type value cannot be parsed + * @since 4.3.2 + */ + public static List parseMediaTypes(@Nullable List mediaTypes) { + if (CollectionUtils.isEmpty(mediaTypes)) { + return Collections.emptyList(); + } else if (mediaTypes.size() == 1) { + return parseMediaTypes(mediaTypes.get(0)); + } else { + List result = new ArrayList<>(8); + for (String mediaType : mediaTypes) { + result.addAll(parseMediaTypes(mediaType)); + } + return result; + } + } + + /** + * Re-create the given mime types as media types. + * + * @since 5.0 + */ + public static List asMediaTypes(List mimeTypes) { + List mediaTypes = new ArrayList<>(mimeTypes.size()); + for (MimeType mimeType : mimeTypes) { + mediaTypes.add(MediaType.asMediaType(mimeType)); + } + return mediaTypes; + } + + /** + * Re-create the given mime type as a media type. + * + * @since 5.0 + */ + public static MediaType asMediaType(MimeType mimeType) { + if (mimeType instanceof MediaType) { + return (MediaType) mimeType; + } + return new MediaType(mimeType.getType(), mimeType.getSubtype(), mimeType.getParameters()); + } + + /** + * Return a string representation of the given list of {@code MediaType} objects. + *

This method can be used to for an {@code Accept} or {@code Content-Type} header. + * + * @param mediaTypes the media types to create a string representation for + * @return the string representation + */ + public static String toString(Collection mediaTypes) { + return MimeTypeUtils.toString(mediaTypes); + } + + /** + * Sorts the given list of {@code MediaType} objects by specificity. + *

Given two media types: + *

    + *
  1. if either media type has a {@linkplain #isWildcardType() wildcard type}, + * then the media type without the wildcard is ordered before the other.
  2. + *
  3. if the two media types have different {@linkplain #getType() types}, + * then they are considered equal and remain their current order.
  4. + *
  5. if either media type has a {@linkplain #isWildcardSubtype() wildcard subtype}, + * then the media type without the wildcard is sorted before the other.
  6. + *
  7. if the two media types have different {@linkplain #getSubtype() subtypes}, + * then they are considered equal and remain their current order.
  8. + *
  9. if the two media types have different {@linkplain #getQualityValue() quality value}, + * then the media type with the highest quality value is ordered before the other.
  10. + *
  11. if the two media types have a different amount of + * {@linkplain #getParameter(String) parameters}, then the + * media type with the most parameters is ordered before the other.
  12. + *
+ *

For example: + *

audio/basic < audio/* < */*
+ *
audio/* < audio/*;q=0.7; audio/*;q=0.3
+ *
audio/basic;level=1 < audio/basic
+ *
audio/basic == text/html
+ *
audio/basic == audio/wave
+ * + * @param mediaTypes the list of media types to be sorted + * @see HTTP 1.1: Semantics + * and Content, section 5.3.2 + */ + public static void sortBySpecificity(List mediaTypes) { + Assert.notNull(mediaTypes, "'mediaTypes' must not be null"); + if (mediaTypes.size() > 1) { + mediaTypes.sort(SPECIFICITY_COMPARATOR); + } + } + + /** + * Sorts the given list of {@code MediaType} objects by quality value. + *

Given two media types: + *

    + *
  1. if the two media types have different {@linkplain #getQualityValue() quality value}, + * then the media type with the highest quality value is ordered before the other.
  2. + *
  3. if either media type has a {@linkplain #isWildcardType() wildcard type}, + * then the media type without the wildcard is ordered before the other.
  4. + *
  5. if the two media types have different {@linkplain #getType() types}, + * then they are considered equal and remain their current order.
  6. + *
  7. if either media type has a {@linkplain #isWildcardSubtype() wildcard subtype}, + * then the media type without the wildcard is sorted before the other.
  8. + *
  9. if the two media types have different {@linkplain #getSubtype() subtypes}, + * then they are considered equal and remain their current order.
  10. + *
  11. if the two media types have a different amount of + * {@linkplain #getParameter(String) parameters}, then the + * media type with the most parameters is ordered before the other.
  12. + *
+ * + * @param mediaTypes the list of media types to be sorted + * @see #getQualityValue() + */ + public static void sortByQualityValue(List mediaTypes) { + Assert.notNull(mediaTypes, "'mediaTypes' must not be null"); + if (mediaTypes.size() > 1) { + mediaTypes.sort(QUALITY_VALUE_COMPARATOR); + } + } + + /** + * Sorts the given list of {@code MediaType} objects by specificity as the + * primary criteria and quality value the secondary. + * + * @see MediaType#sortBySpecificity(List) + * @see MediaType#sortByQualityValue(List) + */ + public static void sortBySpecificityAndQuality(List mediaTypes) { + Assert.notNull(mediaTypes, "'mediaTypes' must not be null"); + if (mediaTypes.size() > 1) { + mediaTypes.sort( + MediaType.SPECIFICITY_COMPARATOR.thenComparing(MediaType.QUALITY_VALUE_COMPARATOR)); + } + } + + @Override + protected void checkParameters(String parameter, String value) { + super.checkParameters(parameter, value); + if (PARAM_QUALITY_FACTOR.equals(parameter)) { + String unquotedValue = unquote(value); + double d = Double.parseDouble(unquotedValue); + Assert.isTrue(d >= 0D && d <= 1D, + () -> "Invalid quality value \"" + unquotedValue + "\": should be between 0.0 and 1.0"); + } + } + + /** + * Return the quality factor, as indicated by a {@code q} parameter, if any. + * Defaults to {@code 1.0}. + * + * @return the quality factor as double value + */ + public double getQualityValue() { + String qualityFactor = getParameter(PARAM_QUALITY_FACTOR); + return (qualityFactor != null ? Double.parseDouble(unquote(qualityFactor)) : 1D); + } + + /** + * Indicate whether this {@code MediaType} includes the given media type. + *

For instance, {@code text/*} includes {@code text/plain} and {@code text/html}, + * and {@code application/*+xml} includes {@code application/soap+xml}, etc. + * This method is not symmetric. + *

Simply calls {@link MimeType#includes(MimeType)} but declared with a + * {@code MediaType} parameter for binary backwards compatibility. + * + * @param other the reference media type with which to compare + * @return {@code true} if this media type includes the given media type; + * {@code false} otherwise + */ + public boolean includes(@Nullable MediaType other) { + return super.includes(other); + } + + /** + * Indicate whether this {@code MediaType} is compatible with the given media type. + *

For instance, {@code text/*} is compatible with {@code text/plain}, + * {@code text/html}, and vice versa. In effect, this method is similar to + * {@link #includes}, except that it is symmetric. + *

Simply calls {@link MimeType#isCompatibleWith(MimeType)} but declared with a + * {@code MediaType} parameter for binary backwards compatibility. + * + * @param other the reference media type with which to compare + * @return {@code true} if this media type is compatible with the given media type; + * {@code false} otherwise + */ + public boolean isCompatibleWith(@Nullable MediaType other) { + return super.isCompatibleWith(other); + } + + /** + * Return a replica of this instance with the quality value of the given {@code MediaType}. + * + * @return the same instance if the given MediaType doesn't have a quality value, + * or a new one otherwise + */ + public MediaType copyQualityValue(MediaType mediaType) { + if (!mediaType.getParameters().containsKey(PARAM_QUALITY_FACTOR)) { + return this; + } + Map params = new LinkedHashMap<>(getParameters()); + params.put(PARAM_QUALITY_FACTOR, mediaType.getParameters().get(PARAM_QUALITY_FACTOR)); + return new MediaType(this, params); + } + + /** + * Return a replica of this instance with its quality value removed. + * + * @return the same instance if the media type doesn't contain a quality value, + * or a new one otherwise + */ + public MediaType removeQualityValue() { + if (!getParameters().containsKey(PARAM_QUALITY_FACTOR)) { + return this; + } + Map params = new LinkedHashMap<>(getParameters()); + params.remove(PARAM_QUALITY_FACTOR); + return new MediaType(this, params); + } + +} \ No newline at end of file diff --git a/framework/src/test/java/org/tron/common/BaseTest.java b/framework/src/test/java/org/tron/common/BaseTest.java index 552808b842c..dd4400e10f2 100644 --- a/framework/src/test/java/org/tron/common/BaseTest.java +++ b/framework/src/test/java/org/tron/common/BaseTest.java @@ -25,6 +25,8 @@ import org.tron.core.config.args.Args; import org.tron.core.db.Manager; import org.tron.core.exception.BalanceInsufficientException; +import org.tron.core.net.peer.PeerConnection; +import org.tron.core.net.peer.PeerManager; import org.tron.core.store.AccountStore; import org.tron.protos.Protocol; @@ -68,6 +70,12 @@ public static void destroy() { Args.clearParam(); } + public void closePeer() { + for (PeerConnection p : PeerManager.getPeers()) { + PeerManager.remove(p.getChannel()); + } + } + public Protocol.Block getSignedBlock(ByteString witness, long time, byte[] privateKey) { long blockTime = System.currentTimeMillis() / 3000 * 3000; if (time != 0) { diff --git a/framework/src/test/java/org/tron/common/EntityTest.java b/framework/src/test/java/org/tron/common/EntityTest.java index 483475a453b..bbdc8631225 100644 --- a/framework/src/test/java/org/tron/common/EntityTest.java +++ b/framework/src/test/java/org/tron/common/EntityTest.java @@ -5,13 +5,16 @@ import static org.junit.Assert.assertTrue; import com.google.common.collect.Lists; +import java.util.ArrayList; import java.util.HashMap; +import java.util.List; import org.apache.commons.collections4.CollectionUtils; import org.junit.Before; import org.junit.Test; import org.tron.common.entity.NodeInfo; import org.tron.common.entity.NodeInfo.MachineInfo; import org.tron.common.entity.NodeInfo.MachineInfo.DeadLockThreadInfo; +import org.tron.common.entity.PeerInfo; public class EntityTest { @@ -54,6 +57,9 @@ public void testDeadLockThreadInfo() { @Test public void testNodeInfo() { + List peerInfoList = new ArrayList<>(); + peerInfoList.add(getDefaultPeerInfo()); + NodeInfo nodeInfo = new NodeInfo(); nodeInfo.setTotalFlow(1L); nodeInfo.setCheatWitnessInfoMap(new HashMap<>()); @@ -62,6 +68,39 @@ public void testNodeInfo() { nodeInfo.setMachineInfo(machineInfo); nodeInfo.setBlock("block"); nodeInfo.setSolidityBlock("solidityBlock"); + nodeInfo.setPeerList(peerInfoList); nodeInfo.transferToProtoEntity(); } + + private PeerInfo getDefaultPeerInfo() { + PeerInfo peerInfo = new PeerInfo(); + peerInfo.setAvgLatency(peerInfo.getAvgLatency()); + peerInfo.setBlockInPorcSize(peerInfo.getBlockInPorcSize()); + peerInfo.setConnectTime(peerInfo.getConnectTime()); + peerInfo.setDisconnectTimes(peerInfo.getDisconnectTimes()); + peerInfo.setHeadBlockTimeWeBothHave(peerInfo.getHeadBlockTimeWeBothHave()); + peerInfo.setHeadBlockWeBothHave(peerInfo.getHeadBlockWeBothHave()); + peerInfo.setHost("host"); + peerInfo.setInFlow(peerInfo.getInFlow()); + peerInfo.setLastBlockUpdateTime(peerInfo.getLastBlockUpdateTime()); + peerInfo.setLastSyncBlock("last"); + peerInfo.setLocalDisconnectReason("localDisconnectReason"); + peerInfo.setNodeCount(peerInfo.getNodeCount()); + peerInfo.setNodeId("nodeId"); + peerInfo.setHeadBlockWeBothHave("headBlockWeBothHave"); + peerInfo.setRemainNum(peerInfo.getRemainNum()); + peerInfo.setRemoteDisconnectReason("remoteDisconnectReason"); + peerInfo.setScore(peerInfo.getScore()); + peerInfo.setPort(peerInfo.getPort()); + peerInfo.setSyncFlag(peerInfo.isSyncFlag()); + peerInfo.setNeedSyncFromPeer(peerInfo.isNeedSyncFromPeer()); + peerInfo.setNeedSyncFromUs(peerInfo.isNeedSyncFromUs()); + peerInfo.setSyncToFetchSize(peerInfo.getSyncToFetchSize()); + peerInfo.setSyncToFetchSizePeekNum(peerInfo.getSyncToFetchSizePeekNum()); + peerInfo.setSyncBlockRequestedSize(peerInfo.getSyncBlockRequestedSize()); + peerInfo.setUnFetchSynNum(peerInfo.getUnFetchSynNum()); + peerInfo.setActive(peerInfo.isActive()); + + return peerInfo; + } } diff --git a/framework/src/test/java/org/tron/common/ParameterTest.java b/framework/src/test/java/org/tron/common/ParameterTest.java index b16be405f61..2f65189ac1c 100644 --- a/framework/src/test/java/org/tron/common/ParameterTest.java +++ b/framework/src/test/java/org/tron/common/ParameterTest.java @@ -129,6 +129,12 @@ public void testCommonParameter() { assertEquals(10, parameter.getMaxConcurrentCallsPerConnection()); parameter.setFlowControlWindow(20); assertEquals(20, parameter.getFlowControlWindow()); + assertEquals(0, parameter.getRpcMaxRstStream()); + parameter.setRpcMaxRstStream(10); + assertEquals(10, parameter.getRpcMaxRstStream()); + assertEquals(0, parameter.getRpcSecondsPerWindow()); + parameter.setRpcSecondsPerWindow(5); + assertEquals(5, parameter.getRpcSecondsPerWindow()); parameter.setMaxConnectionIdleInMillis(1000); assertEquals(1000, parameter.getMaxConnectionIdleInMillis()); parameter.setBlockProducedTimeOut(500); diff --git a/framework/src/test/java/org/tron/common/backup/BackupServerTest.java b/framework/src/test/java/org/tron/common/backup/BackupServerTest.java index c40aca7e17a..18e264eead2 100644 --- a/framework/src/test/java/org/tron/common/backup/BackupServerTest.java +++ b/framework/src/test/java/org/tron/common/backup/BackupServerTest.java @@ -10,6 +10,7 @@ import org.junit.rules.Timeout; import org.tron.common.backup.socket.BackupServer; import org.tron.common.parameter.CommonParameter; +import org.tron.common.utils.PublicMethod; import org.tron.core.Constant; import org.tron.core.config.args.Args; @@ -26,7 +27,7 @@ public class BackupServerTest { @Before public void setUp() throws Exception { Args.setParam(new String[]{"-d", temporaryFolder.newFolder().toString()}, Constant.TEST_CONF); - CommonParameter.getInstance().setBackupPort(80); + CommonParameter.getInstance().setBackupPort(PublicMethod.chooseRandomPort()); List members = new ArrayList<>(); members.add("127.0.0.2"); CommonParameter.getInstance().setBackupMembers(members); diff --git a/framework/src/test/java/org/tron/common/crypto/ECKeyTest.java b/framework/src/test/java/org/tron/common/crypto/ECKeyTest.java index 4e7d45ee8d7..273672e8342 100644 --- a/framework/src/test/java/org/tron/common/crypto/ECKeyTest.java +++ b/framework/src/test/java/org/tron/common/crypto/ECKeyTest.java @@ -5,6 +5,7 @@ import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertNotEquals; import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; import static org.junit.Assert.assertTrue; import static org.junit.Assert.fail; import static org.tron.common.utils.client.utils.AbiUtil.generateOccupationConstantPrivateKey; @@ -67,6 +68,11 @@ public void testFromPrivateKey() { assertTrue(key.isPubKeyCanonical()); assertTrue(key.hasPrivKey()); assertArrayEquals(pubKey, key.getPubKey()); + + key = ECKey.fromPrivate((byte[]) null); + assertNull(key); + key = ECKey.fromPrivate(new byte[0]); + assertNull(key); } @Test(expected = IllegalArgumentException.class) diff --git a/framework/src/test/java/org/tron/common/crypto/SM2KeyTest.java b/framework/src/test/java/org/tron/common/crypto/SM2KeyTest.java index b84026d2085..87e4e14698c 100644 --- a/framework/src/test/java/org/tron/common/crypto/SM2KeyTest.java +++ b/framework/src/test/java/org/tron/common/crypto/SM2KeyTest.java @@ -4,6 +4,7 @@ import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; import static org.junit.Assert.assertTrue; import static org.junit.Assert.fail; import static org.tron.common.utils.client.utils.AbiUtil.generateOccupationConstantPrivateKey; @@ -64,6 +65,11 @@ public void testFromPrivateKey() { assertTrue(key.isPubKeyCanonical()); assertTrue(key.hasPrivKey()); assertArrayEquals(pubKey, key.getPubKey()); + + key = SM2.fromPrivate((byte[]) null); + assertNull(key); + key = SM2.fromPrivate(new byte[0]); + assertNull(key); } @Test(expected = IllegalArgumentException.class) diff --git a/framework/src/test/java/org/tron/common/logsfilter/NativeMessageQueueTest.java b/framework/src/test/java/org/tron/common/logsfilter/NativeMessageQueueTest.java index e6bb407bb53..d356e43d66c 100644 --- a/framework/src/test/java/org/tron/common/logsfilter/NativeMessageQueueTest.java +++ b/framework/src/test/java/org/tron/common/logsfilter/NativeMessageQueueTest.java @@ -55,18 +55,19 @@ public void publishTrigger() { public void startSubscribeThread() { Thread thread = new Thread(() -> { - ZContext context = new ZContext(); - ZMQ.Socket subscriber = context.createSocket(SocketType.SUB); + try (ZContext context = new ZContext()) { + ZMQ.Socket subscriber = context.createSocket(SocketType.SUB); - Assert.assertEquals(true, subscriber.connect(String.format("tcp://localhost:%d", bindPort))); - Assert.assertEquals(true, subscriber.subscribe(topic)); + Assert.assertTrue(subscriber.connect(String.format("tcp://localhost:%d", bindPort))); + Assert.assertTrue(subscriber.subscribe(topic)); - while (!Thread.currentThread().isInterrupted()) { - byte[] message = subscriber.recv(); - String triggerMsg = new String(message); - - Assert.assertEquals(true, triggerMsg.contains(dataToSend) || triggerMsg.contains(topic)); + while (!Thread.currentThread().isInterrupted()) { + byte[] message = subscriber.recv(); + String triggerMsg = new String(message); + Assert.assertTrue(triggerMsg.contains(dataToSend) || triggerMsg.contains(topic)); + } + // ZMQ.Socket will be automatically closed when ZContext is closed } }); thread.start(); diff --git a/framework/src/test/java/org/tron/common/runtime/vm/BatchValidateSignContractTest.java b/framework/src/test/java/org/tron/common/runtime/vm/BatchValidateSignContractTest.java index fc60d8c0648..c18eb396546 100644 --- a/framework/src/test/java/org/tron/common/runtime/vm/BatchValidateSignContractTest.java +++ b/framework/src/test/java/org/tron/common/runtime/vm/BatchValidateSignContractTest.java @@ -15,6 +15,7 @@ import org.tron.core.db.TransactionTrace; import org.tron.core.vm.PrecompiledContracts; import org.tron.core.vm.PrecompiledContracts.BatchValidateSign; +import org.tron.core.vm.config.VMConfig; @Slf4j @@ -74,6 +75,13 @@ public void staticCallTest() { ret = validateMultiSign(hash, signatures, addresses); Assert.assertEquals(ret.getValue().length, 32); Assert.assertArrayEquals(ret.getValue(), new byte[32]); + + //after optimized + VMConfig.initAllowTvmSelfdestructRestriction(1); + ret = validateMultiSign(hash, signatures, addresses); + Assert.assertEquals(ret.getValue().length, 32); + Assert.assertArrayEquals(ret.getValue(), new byte[32]); + VMConfig.initAllowTvmSelfdestructRestriction(0); System.gc(); // force triggering full gc to avoid timeout for next test } diff --git a/framework/src/test/java/org/tron/common/runtime/vm/Create2Test.java b/framework/src/test/java/org/tron/common/runtime/vm/Create2Test.java index f400b3215ee..6fa2801c51f 100644 --- a/framework/src/test/java/org/tron/common/runtime/vm/Create2Test.java +++ b/framework/src/test/java/org/tron/common/runtime/vm/Create2Test.java @@ -19,9 +19,9 @@ import org.tron.core.Wallet; import org.tron.core.exception.ContractExeException; import org.tron.core.exception.ContractValidateException; -import org.tron.core.exception.JsonRpcInvalidParamsException; import org.tron.core.exception.ReceiptCheckErrException; import org.tron.core.exception.VMIllegalException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; import org.tron.core.services.NodeInfoService; import org.tron.core.services.jsonrpc.TronJsonRpcImpl; import org.tron.core.vm.config.ConfigLoader; diff --git a/framework/src/test/java/org/tron/common/runtime/vm/OperationsTest.java b/framework/src/test/java/org/tron/common/runtime/vm/OperationsTest.java index 3315005b7d2..d6bbdddc854 100644 --- a/framework/src/test/java/org/tron/common/runtime/vm/OperationsTest.java +++ b/framework/src/test/java/org/tron/common/runtime/vm/OperationsTest.java @@ -1,6 +1,8 @@ package org.tron.common.runtime.vm; import static org.junit.Assert.assertEquals; +import static org.mockito.ArgumentMatchers.any; +import static org.tron.core.config.Parameter.ChainConstant.FROZEN_PERIOD; import java.util.List; import java.util.Random; @@ -13,25 +15,33 @@ import org.junit.BeforeClass; import org.junit.Ignore; import org.junit.Test; +import org.mockito.Mockito; +import org.springframework.beans.factory.annotation.Autowired; import org.springframework.util.StringUtils; import org.tron.common.BaseTest; import org.tron.common.parameter.CommonParameter; import org.tron.common.runtime.InternalTransaction; import org.tron.common.utils.DecodeUtil; import org.tron.core.Constant; +import org.tron.core.Wallet; +import org.tron.core.capsule.AccountCapsule; import org.tron.core.config.args.Args; import org.tron.core.exception.ContractValidateException; import org.tron.core.store.StoreFactory; import org.tron.core.vm.EnergyCost; import org.tron.core.vm.JumpTable; +import org.tron.core.vm.MessageCall; import org.tron.core.vm.Op; import org.tron.core.vm.Operation; +import org.tron.core.vm.OperationActions; import org.tron.core.vm.OperationRegistry; +import org.tron.core.vm.PrecompiledContracts; import org.tron.core.vm.VM; import org.tron.core.vm.config.ConfigLoader; import org.tron.core.vm.config.VMConfig; import org.tron.core.vm.program.Program; import org.tron.core.vm.program.invoke.ProgramInvokeMockImpl; +import org.tron.core.vm.repository.Repository; import org.tron.protos.Protocol; @Slf4j @@ -40,6 +50,8 @@ public class OperationsTest extends BaseTest { private ProgramInvokeMockImpl invoke; private Program program; private final JumpTable jumpTable = OperationRegistry.getTable(); + @Autowired + private Wallet wallet; @BeforeClass public static void init() { @@ -749,6 +761,30 @@ public void testPushDupSwapAndLogOperations() throws ContractValidateException { Assert.assertEquals(2158, program.getResult().getEnergyUsed()); } + @Test + public void testCallOperations() throws ContractValidateException { + invoke = new ProgramInvokeMockImpl(); + Protocol.Transaction trx = Protocol.Transaction.getDefaultInstance(); + InternalTransaction interTrx = + new InternalTransaction(trx, InternalTransaction.TrxType.TRX_UNKNOWN_TYPE); + + byte prePrefixByte = DecodeUtil.addressPreFixByte; + DecodeUtil.addressPreFixByte = Constant.ADD_PRE_FIX_BYTE_MAINNET; + VMConfig.initAllowTvmSelfdestructRestriction(1); + + program = new Program(new byte[0], new byte[0], invoke, interTrx); + MessageCall messageCall = new MessageCall( + Op.CALL, new DataWord(10000), + DataWord.ZERO(), DataWord.ZERO(), + DataWord.ZERO(), DataWord.ZERO(), + DataWord.ZERO(), DataWord.ZERO(), + DataWord.ZERO(), false); + program.callToPrecompiledAddress(messageCall, new PrecompiledContracts.ECRecover()); + + DecodeUtil.addressPreFixByte = prePrefixByte; + VMConfig.initAllowTvmSelfdestructRestriction(0); + } + @Test public void testOtherOperations() throws ContractValidateException { invoke = new ProgramInvokeMockImpl(); @@ -886,6 +922,12 @@ public void testSuicideCost() throws ContractValidateException { Assert.assertEquals(25000, EnergyCost.getSuicideCost2(program)); invoke.getDeposit().createAccount(receiver2, Protocol.AccountType.Normal); Assert.assertEquals(0, EnergyCost.getSuicideCost2(program)); + + byte[] receiver3 = generateRandomAddress(); + program.stackPush(new DataWord(receiver3)); + Assert.assertEquals(30000, EnergyCost.getSuicideCost3(program)); + invoke.getDeposit().createAccount(receiver3, Protocol.AccountType.Normal); + Assert.assertEquals(5000, EnergyCost.getSuicideCost3(program)); } @Test @@ -911,6 +953,85 @@ public void testSuicideAction() throws ContractValidateException { VMConfig.initAllowEnergyAdjustment(0); } + @Test + public void testCanSuicide2() throws ContractValidateException { + VMConfig.initAllowTvmFreeze(1); + VMConfig.initAllowTvmFreezeV2(1); + + byte[] contractAddr = Hex.decode("41471fd3ad3e9eeadeec4608b92d16ce6b500704cc"); + invoke = new ProgramInvokeMockImpl(StoreFactory.getInstance(), new byte[0], contractAddr); + + program = new Program(null, null, invoke, + new InternalTransaction( + Protocol.Transaction.getDefaultInstance(), + InternalTransaction.TrxType.TRX_UNKNOWN_TYPE)); + program.getContractState().createAccount( + program.getContextAddress(), Protocol.AccountType.Contract); + Assert.assertTrue(program.canSuicide2()); + + long nowInMs = + program.getContractState().getDynamicPropertiesStore().getLatestBlockHeaderTimestamp(); + long expireTime = nowInMs + FROZEN_PERIOD; + AccountCapsule owner = program.getContractState().getAccount(program.getContextAddress()); + owner.setFrozenForEnergy(1000000, expireTime); + program.getContractState().updateAccount(program.getContextAddress(), owner); + Assert.assertFalse(program.canSuicide2()); + + VMConfig.initAllowTvmFreeze(0); + VMConfig.initAllowTvmFreezeV2(0); + } + + @Test + public void testSuicideAction2() throws ContractValidateException { + byte[] contractAddr = Hex.decode("41471fd3ad3e9eeadeec4608b92d16ce6b500704cc"); + invoke = new ProgramInvokeMockImpl(StoreFactory.getInstance(), new byte[0], contractAddr); + Assert.assertTrue(invoke.getDeposit().isNewContract(contractAddr)); + + program = new Program(null, null, invoke, + new InternalTransaction( + Protocol.Transaction.getDefaultInstance(), + InternalTransaction.TrxType.TRX_UNKNOWN_TYPE)); + + VMConfig.initAllowEnergyAdjustment(1); + VMConfig.initAllowTvmSelfdestructRestriction(1); + VMConfig.initAllowTvmFreeze(1); + VMConfig.initAllowTvmFreezeV2(1); + VMConfig.initAllowTvmCompatibleEvm(1); + VMConfig.initAllowTvmVote(1); + byte prePrefixByte = DecodeUtil.addressPreFixByte; + DecodeUtil.addressPreFixByte = Constant.ADD_PRE_FIX_BYTE_MAINNET; + + program.stackPush(new DataWord( + dbManager.getAccountStore().getBlackhole().getAddress().toByteArray())); + OperationActions.suicideAction2(program); + + Assert.assertEquals(1, program.getResult().getDeleteAccounts().size()); + + + invoke = new ProgramInvokeMockImpl(StoreFactory.getInstance(), new byte[0], contractAddr); + program = new Program(null, null, invoke, + new InternalTransaction( + Protocol.Transaction.getDefaultInstance(), + InternalTransaction.TrxType.TRX_UNKNOWN_TYPE)); + Program spyProgram = Mockito.spy(program); + Repository realContractState = program.getContractState(); + Repository spyContractState = Mockito.spy(realContractState); + Mockito.when(spyContractState.isNewContract(any(byte[].class))).thenReturn(false); + Mockito.when(spyProgram.getContractState()).thenReturn(spyContractState); + spyProgram.suicide2(new DataWord( + dbManager.getAccountStore().getBlackhole().getAddress().toByteArray())); + + Assert.assertEquals(0, spyProgram.getResult().getDeleteAccounts().size()); + + DecodeUtil.addressPreFixByte = prePrefixByte; + VMConfig.initAllowEnergyAdjustment(0); + VMConfig.initAllowTvmSelfdestructRestriction(0); + VMConfig.initAllowTvmFreeze(0); + VMConfig.initAllowTvmFreezeV2(0); + VMConfig.initAllowTvmCompatibleEvm(0); + VMConfig.initAllowTvmVote(0); + } + @Test public void testVoteWitnessCost() throws ContractValidateException { // Build stack environment, the stack from top to bottom is 0x00, 0x80, 0x00, 0x80 diff --git a/framework/src/test/java/org/tron/common/runtime/vm/ValidateMultiSignContractTest.java b/framework/src/test/java/org/tron/common/runtime/vm/ValidateMultiSignContractTest.java index a688f5f9a29..894022fcea1 100644 --- a/framework/src/test/java/org/tron/common/runtime/vm/ValidateMultiSignContractTest.java +++ b/framework/src/test/java/org/tron/common/runtime/vm/ValidateMultiSignContractTest.java @@ -25,6 +25,7 @@ import org.tron.core.config.args.Args; import org.tron.core.store.StoreFactory; import org.tron.core.vm.PrecompiledContracts.ValidateMultiSign; +import org.tron.core.vm.config.VMConfig; import org.tron.core.vm.repository.Repository; import org.tron.core.vm.repository.RepositoryImpl; import org.tron.protos.Protocol; @@ -123,6 +124,13 @@ public void testDifferentCase() { validateMultiSign(StringUtil.encode58Check(key.getAddress()), permissionId, data, signs) .getValue(), DataWord.ONE().getData()); + //after optimized + VMConfig.initAllowTvmSelfdestructRestriction(1); + Assert.assertArrayEquals( + validateMultiSign(StringUtil.encode58Check(key.getAddress()), permissionId, data, signs) + .getValue(), DataWord.ONE().getData()); + VMConfig.initAllowTvmSelfdestructRestriction(0); + //weight not enough signs = new ArrayList<>(); signs.add(Hex.toHexString(key1.sign(toSign).toByteArray())); diff --git a/framework/src/test/java/org/tron/common/storage/CheckOrInitEngineTest.java b/framework/src/test/java/org/tron/common/storage/CheckOrInitEngineTest.java new file mode 100644 index 00000000000..90aac10c0b6 --- /dev/null +++ b/framework/src/test/java/org/tron/common/storage/CheckOrInitEngineTest.java @@ -0,0 +1,263 @@ +package org.tron.common.storage; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertThrows; +import static org.junit.Assert.assertTrue; +import static org.mockito.Mockito.mockStatic; +import static org.tron.core.db.common.DbSourceInter.ENGINE_FILE; +import static org.tron.core.db.common.DbSourceInter.ENGINE_KEY; +import static org.tron.core.db.common.DbSourceInter.LEVELDB; +import static org.tron.core.db.common.DbSourceInter.ROCKSDB; +import static org.tron.core.db.common.DbSourceInter.checkOrInitEngine; + +import com.google.common.base.Strings; +import java.io.File; +import java.io.IOException; +import java.nio.file.Paths; +import org.junit.After; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.mockito.MockedStatic; +import org.mockito.Mockito; +import org.tron.common.utils.FileUtil; +import org.tron.common.utils.PropUtil; +import org.tron.core.exception.TronError; + + +public class CheckOrInitEngineTest { + + @Rule + public TemporaryFolder temporaryFolder = new TemporaryFolder(); + + private static final String ACCOUNT = "account"; + + @After + public void clearMocks() { + Mockito.clearAllCaches(); + } + + @Test + public void testLevelDbDetectedWhenExpectingRocksDb() throws IOException { + try (MockedStatic fileUtil = mockStatic(FileUtil.class); + MockedStatic propUtil = mockStatic(PropUtil.class); + MockedStatic strings = mockStatic(Strings.class)) { + + String dir = temporaryFolder.newFolder(ACCOUNT).toString(); + File currentFile = new File(dir, "CURRENT"); + assertTrue(currentFile.createNewFile()); + TronError.ErrCode errCode = TronError.ErrCode.ROCKSDB_INIT; + TronError exception = assertThrows(TronError.class, () -> + checkOrInitEngine(ROCKSDB, dir, errCode)); + assertEquals("Cannot open LEVELDB database with ROCKSDB engine.", + exception.getMessage()); + assertEquals(errCode, exception.getErrCode()); + } + } + + @Test + public void testCannotCreateDir() { + try (MockedStatic fileUtil = mockStatic(FileUtil.class)) { + String dir = "/invalid/path/that/cannot/be/created"; + + fileUtil.when(() -> FileUtil.createDirIfNotExists(dir)).thenReturn(false); + TronError.ErrCode errCode = TronError.ErrCode.LEVELDB_INIT; + TronError exception = assertThrows(TronError.class, () -> + checkOrInitEngine(LEVELDB, dir, errCode)); + assertEquals("Cannot create dir: " + dir + ".", exception.getMessage()); + assertEquals(errCode, exception.getErrCode()); + } + } + + @Test + public void testCannotCreateEngineFile() throws IOException { + try (MockedStatic fileUtil = mockStatic(FileUtil.class)) { + String dir = temporaryFolder.newFolder().toString(); + String engineFile = Paths.get(dir, ENGINE_FILE).toString(); + fileUtil.when(() -> FileUtil.createDirIfNotExists(dir)).thenReturn(true); + fileUtil.when(() -> FileUtil.createFileIfNotExists(engineFile)).thenReturn(false); + TronError.ErrCode errCode = TronError.ErrCode.ROCKSDB_INIT; + TronError exception = assertThrows(TronError.class, () -> + checkOrInitEngine(ROCKSDB, dir, errCode)); + + assertEquals("Cannot create file: " + engineFile + ".", exception.getMessage()); + assertEquals(errCode, exception.getErrCode()); + } + } + + @Test + public void testCannotWritePropertyFile() throws IOException { + try (MockedStatic fileUtil = mockStatic(FileUtil.class); + MockedStatic propUtil = mockStatic(PropUtil.class); + MockedStatic strings = mockStatic(Strings.class)) { + + String dir = temporaryFolder.newFolder().toString(); + String engineFile = Paths.get(dir, ENGINE_FILE).toString(); + + fileUtil.when(() -> FileUtil.createDirIfNotExists(dir)).thenReturn(true); + fileUtil.when(() -> FileUtil.createFileIfNotExists(engineFile)).thenReturn(true); + + propUtil.when(() -> PropUtil.readProperty(engineFile, ENGINE_KEY)).thenReturn(null); + strings.when(() -> Strings.isNullOrEmpty(null)).thenReturn(true); + + propUtil.when(() -> PropUtil.writeProperty(engineFile, ENGINE_KEY, ROCKSDB)) + .thenReturn(false); + + TronError.ErrCode errCode = TronError.ErrCode.LEVELDB_INIT; + + TronError exception = assertThrows(TronError.class, () -> + checkOrInitEngine(ROCKSDB, dir, errCode)); + + assertEquals("Cannot write file: " + engineFile + ".", exception.getMessage()); + assertEquals(errCode, exception.getErrCode()); + } + + } + + @Test + public void testEngineMismatch() throws IOException { + try (MockedStatic fileUtil = mockStatic(FileUtil.class); + MockedStatic propUtil = mockStatic(PropUtil.class); + MockedStatic strings = mockStatic(Strings.class)) { + + String dir = temporaryFolder.newFolder(ACCOUNT).toString(); + String engineFile = Paths.get(dir, ENGINE_FILE).toString(); + + fileUtil.when(() -> FileUtil.createDirIfNotExists(dir)).thenReturn(true); + fileUtil.when(() -> FileUtil.createFileIfNotExists(engineFile)).thenReturn(true); + + propUtil.when(() -> PropUtil.readProperty(engineFile, ENGINE_KEY)).thenReturn(LEVELDB); + strings.when(() -> Strings.isNullOrEmpty(LEVELDB)).thenReturn(false); + + TronError.ErrCode errCode = TronError.ErrCode.ROCKSDB_INIT; + + TronError exception = assertThrows(TronError.class, () -> + checkOrInitEngine(ROCKSDB, dir, errCode)); + + assertEquals("Cannot open LEVELDB database with ROCKSDB engine.", + exception.getMessage()); + assertEquals(errCode, exception.getErrCode()); + } + } + + @Test + public void testSuccessfulFirstTimeInit() throws IOException { + try (MockedStatic fileUtil = mockStatic(FileUtil.class); + MockedStatic propUtil = mockStatic(PropUtil.class); + MockedStatic strings = mockStatic(Strings.class)) { + + String dir = temporaryFolder.newFolder(ACCOUNT).toString(); + String engineFile = Paths.get(dir, ENGINE_FILE).toString(); + + fileUtil.when(() -> FileUtil.createDirIfNotExists(dir)).thenReturn(true); + fileUtil.when(() -> FileUtil.createFileIfNotExists(engineFile)).thenReturn(true); + + propUtil.when(() -> PropUtil.readProperty(engineFile, ENGINE_KEY)) + .thenReturn(null) + .thenReturn(LEVELDB); + strings.when(() -> Strings.isNullOrEmpty(null)).thenReturn(true); + + propUtil.when(() -> PropUtil.writeProperty(engineFile, ENGINE_KEY, LEVELDB)) + .thenReturn(true); + + TronError.ErrCode errCode = TronError.ErrCode.LEVELDB_INIT; + checkOrInitEngine(LEVELDB, dir, errCode); + } + } + + @Test + public void testSuccessfulExistingEngine() throws IOException { + try (MockedStatic fileUtil = mockStatic(FileUtil.class); + MockedStatic propUtil = mockStatic(PropUtil.class); + MockedStatic strings = mockStatic(Strings.class)) { + + String dir = temporaryFolder.newFolder(ACCOUNT).toString(); + String engineFile = Paths.get(dir, ENGINE_FILE).toString(); + + fileUtil.when(() -> FileUtil.createDirIfNotExists(dir)).thenReturn(true); + fileUtil.when(() -> FileUtil.createFileIfNotExists(engineFile)).thenReturn(true); + propUtil.when(() -> PropUtil.readProperty(engineFile, ENGINE_KEY)).thenReturn(ROCKSDB); + strings.when(() -> Strings.isNullOrEmpty(ROCKSDB)).thenReturn(false); + + TronError.ErrCode errCode = TronError.ErrCode.ROCKSDB_INIT; + checkOrInitEngine(ROCKSDB, dir, errCode); + } + } + + @Test + /** + * 000003.log CURRENT LOCK MANIFEST-000002 + */ + public void testCurrentFileExistsWithNoEngineFile() throws IOException { + try (MockedStatic fileUtil = mockStatic(FileUtil.class); + MockedStatic propUtil = mockStatic(PropUtil.class); + MockedStatic strings = mockStatic(Strings.class)) { + + String dir = temporaryFolder.newFolder(ACCOUNT).toString(); + String engineFile = Paths.get(dir, ENGINE_FILE).toString(); + File currentFile = new File(dir, "CURRENT"); + assertTrue(currentFile.createNewFile()); + + fileUtil.when(() -> FileUtil.createDirIfNotExists(dir)).thenReturn(true); + fileUtil.when(() -> FileUtil.createFileIfNotExists(engineFile)).thenReturn(true); + propUtil.when(() -> PropUtil.readProperty(engineFile, ENGINE_KEY)).thenReturn(LEVELDB); + strings.when(() -> Strings.isNullOrEmpty(LEVELDB)).thenReturn(false); + + TronError.ErrCode errCode = TronError.ErrCode.LEVELDB_INIT; + + checkOrInitEngine(LEVELDB, dir, errCode); + } + } + + @Test + /** + * 000003.log CURRENT LOCK MANIFEST-000002 engine.properties(RocksDB) + */ + public void testCurrentFileExistsEngineFileExists() throws IOException { + try (MockedStatic fileUtil = mockStatic(FileUtil.class); + MockedStatic propUtil = mockStatic(PropUtil.class); + MockedStatic strings = mockStatic(Strings.class)) { + + String dir = temporaryFolder.newFolder(ACCOUNT).toString(); + String engineFile = Paths.get(dir, ENGINE_FILE).toString(); + + File currentFile = new File(dir, "CURRENT"); + File engineFileObj = new File(engineFile); + assertTrue(currentFile.createNewFile()); + assertTrue(engineFileObj.createNewFile()); + + + fileUtil.when(() -> FileUtil.createDirIfNotExists(dir)).thenReturn(true); + fileUtil.when(() -> FileUtil.createFileIfNotExists(engineFile)).thenReturn(true); + + propUtil.when(() -> PropUtil.readProperty(engineFile, ENGINE_KEY)).thenReturn(ROCKSDB); + strings.when(() -> Strings.isNullOrEmpty(ROCKSDB)).thenReturn(false); + + TronError.ErrCode errCode = TronError.ErrCode.ROCKSDB_INIT; + checkOrInitEngine(ROCKSDB, dir, errCode); + } + } + + @Test + public void testEmptyStringEngine() throws IOException { + try (MockedStatic fileUtil = mockStatic(FileUtil.class); + MockedStatic propUtil = mockStatic(PropUtil.class); + MockedStatic strings = mockStatic(Strings.class)) { + + String dir = temporaryFolder.newFolder("account").toString(); + String engineFile = Paths.get(dir, ENGINE_FILE).toString(); + + fileUtil.when(() -> FileUtil.createDirIfNotExists(dir)).thenReturn(true); + fileUtil.when(() -> FileUtil.createFileIfNotExists(engineFile)).thenReturn(true); + + propUtil.when(() -> PropUtil.readProperty(engineFile, ENGINE_KEY)) + .thenReturn("").thenReturn(ROCKSDB); + strings.when(() -> Strings.isNullOrEmpty("")).thenReturn(true); + + propUtil.when(() -> PropUtil.writeProperty(engineFile, ENGINE_KEY, ROCKSDB)) + .thenReturn(true); + TronError.ErrCode errCode = TronError.ErrCode.ROCKSDB_INIT; + checkOrInitEngine(ROCKSDB, dir, errCode); + } + } +} diff --git a/framework/src/test/java/org/tron/common/storage/leveldb/LevelDbDataSourceImplTest.java b/framework/src/test/java/org/tron/common/storage/leveldb/LevelDbDataSourceImplTest.java index bf18b988f19..78cbba3d079 100644 --- a/framework/src/test/java/org/tron/common/storage/leveldb/LevelDbDataSourceImplTest.java +++ b/framework/src/test/java/org/tron/common/storage/leveldb/LevelDbDataSourceImplTest.java @@ -28,6 +28,7 @@ import com.google.common.collect.Sets; import java.io.File; import java.io.IOException; +import java.nio.file.Paths; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; @@ -38,15 +39,24 @@ import java.util.Set; import java.util.stream.Collectors; import lombok.extern.slf4j.Slf4j; +import org.iq80.leveldb.DBException; import org.junit.AfterClass; import org.junit.Assert; import org.junit.Before; import org.junit.ClassRule; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.ExpectedException; import org.junit.rules.TemporaryFolder; +import org.rocksdb.RocksDB; +import org.tron.common.parameter.CommonParameter; +import org.tron.common.storage.WriteOptionsWrapper; +import org.tron.common.storage.rocksdb.RocksDbDataSourceImpl; import org.tron.common.utils.ByteArray; import org.tron.common.utils.FileUtil; +import org.tron.common.utils.PropUtil; import org.tron.common.utils.PublicMethod; +import org.tron.common.utils.StorageUtils; import org.tron.core.Constant; import org.tron.core.config.args.Args; import org.tron.core.db2.common.WrappedByteArray; @@ -73,6 +83,14 @@ public class LevelDbDataSourceImplTest { private byte[] key5 = "00000005aa".getBytes(); private byte[] key6 = "00000006aa".getBytes(); + + @Rule + public final ExpectedException exception = ExpectedException.none(); + + static { + RocksDB.loadLibrary(); + } + /** * Release resources. */ @@ -94,7 +112,6 @@ public void testPutGet() { dataSourceTest.resetDb(); String key1 = PublicMethod.getRandomPrivateKey(); byte[] key = key1.getBytes(); - dataSourceTest.initDB(); String value1 = "50000"; byte[] value = value1.getBytes(); @@ -102,8 +119,19 @@ public void testPutGet() { assertNotNull(dataSourceTest.getData(key)); assertEquals(1, dataSourceTest.allKeys().size()); + assertEquals(1, dataSourceTest.getTotal()); + assertEquals(1, dataSourceTest.allValues().size()); assertEquals("50000", ByteArray.toStr(dataSourceTest.getData(key1.getBytes()))); + dataSourceTest.deleteData(key); + assertNull(dataSourceTest.getData(key)); + assertEquals(0, dataSourceTest.getTotal()); + dataSourceTest.iterator().forEachRemaining(entry -> Assert.fail("iterator should be empty")); + dataSourceTest.stream().forEach(entry -> Assert.fail("stream should be empty")); + dataSourceTest.stat(); dataSourceTest.closeDB(); + dataSourceTest.stat(); // stat again + exception.expect(DBException.class); + dataSourceTest.deleteData(key); } @Test @@ -126,8 +154,6 @@ public void testReset() { public void testupdateByBatchInner() { LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_updateByBatch"); - dataSource.initDB(); - dataSource.resetDb(); String key1 = PublicMethod.getRandomPrivateKey(); String value1 = "50000"; String key2 = PublicMethod.getRandomPrivateKey(); @@ -142,6 +168,25 @@ public void testupdateByBatchInner() { assertEquals("50000", ByteArray.toStr(dataSource.getData(key1.getBytes()))); assertEquals("10000", ByteArray.toStr(dataSource.getData(key2.getBytes()))); assertEquals(2, dataSource.allKeys().size()); + + rows.clear(); + rows.put(key1.getBytes(), null); + rows.put(key2.getBytes(), null); + try (WriteOptionsWrapper options = WriteOptionsWrapper.getInstance()) { + dataSource.updateByBatch(rows, options); + } + assertEquals(0, dataSource.allKeys().size()); + + rows.clear(); + rows.put(key1.getBytes(), value1.getBytes()); + rows.put(key2.getBytes(), null); + dataSource.updateByBatch(rows); + assertEquals("50000", ByteArray.toStr(dataSource.getData(key1.getBytes()))); + assertEquals(1, dataSource.allKeys().size()); + rows.clear(); + rows.put(null, null); + exception.expect(RuntimeException.class); + dataSource.updateByBatch(rows); dataSource.closeDB(); } @@ -149,7 +194,6 @@ public void testupdateByBatchInner() { public void testdeleteData() { LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_delete"); - dataSource.initDB(); String key1 = PublicMethod.getRandomPrivateKey(); byte[] key = key1.getBytes(); dataSource.deleteData(key); @@ -163,8 +207,6 @@ public void testdeleteData() { public void testallKeys() { LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_find_key"); - dataSource.initDB(); - dataSource.resetDb(); String key1 = PublicMethod.getRandomPrivateKey(); byte[] key = key1.getBytes(); @@ -187,7 +229,6 @@ public void testallKeys() { @Test(timeout = 1000) public void testLockReleased() { - dataSourceTest.initDB(); // normal close dataSourceTest.closeDB(); // closing already closed db. @@ -202,8 +243,6 @@ public void testLockReleased() { public void allKeysTest() { LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_allKeysTest_key"); - dataSource.initDB(); - dataSource.resetDb(); byte[] key = "0000000987b10fbb7f17110757321".getBytes(); byte[] value = "50000".getBytes(); @@ -216,7 +255,6 @@ public void allKeysTest() { logger.info(ByteArray.toStr(keyOne)); }); assertEquals(2, dataSource.allKeys().size()); - dataSource.resetDb(); dataSource.closeDB(); } @@ -242,26 +280,10 @@ private void putSomeKeyValue(LevelDbDataSourceImpl dataSource) { dataSource.putData(key4, value4); } - @Test - public void seekTest() { - LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( - Args.getInstance().getOutputDirectory(), "test_seek_key"); - dataSource.initDB(); - dataSource.resetDb(); - - putSomeKeyValue(dataSource); - Assert.assertTrue(true); - dataSource.resetDb(); - dataSource.closeDB(); - } - @Test public void getValuesNext() { LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_getValuesNext_key"); - dataSource.initDB(); - dataSource.resetDb(); - putSomeKeyValue(dataSource); Set seekKeyLimitNext = dataSource.getValuesNext("0000000300".getBytes(), 2); HashSet hashSet = Sets.newHashSet(ByteArray.toStr(value3), ByteArray.toStr(value4)); @@ -276,7 +298,6 @@ public void getValuesNext() { public void testGetTotal() { LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_getTotal_key"); - dataSource.initDB(); dataSource.resetDb(); Map dataMapset = Maps.newHashMap(); @@ -293,8 +314,6 @@ public void testGetTotal() { public void getKeysNext() { LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_getKeysNext_key"); - dataSource.initDB(); - dataSource.resetDb(); putSomeKeyValue(dataSource); int limit = 2; @@ -304,8 +323,6 @@ public void getKeysNext() { for (int i = 0; i < limit; i++) { Assert.assertArrayEquals(list.get(i), seekKeyLimitNext.get(i)); } - - dataSource.resetDb(); dataSource.closeDB(); } @@ -313,9 +330,6 @@ public void getKeysNext() { public void prefixQueryTest() { LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_prefixQuery"); - dataSource.initDB(); - dataSource.resetDb(); - putSomeKeyValue(dataSource); // put a kv that will not be queried. byte[] key7 = "0000001".getBytes(); @@ -341,23 +355,113 @@ public void prefixQueryTest() { Assert.assertEquals(list.size(), result.size()); list.forEach(entry -> Assert.assertTrue(result.contains(entry))); - dataSource.resetDb(); dataSource.closeDB(); } @Test public void initDbTest() { makeExceptionDb("test_initDb"); - LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( - Args.getInstance().getOutputDirectory(), "test_initDb"); - TronError thrown = assertThrows(TronError.class, dataSource::initDB); + TronError thrown = assertThrows(TronError.class, () -> new LevelDbDataSourceImpl( + Args.getInstance().getOutputDirectory(), "test_initDb")); assertEquals(TronError.ErrCode.LEVELDB_INIT, thrown.getErrCode()); } + @Test + public void testCheckOrInitEngine() { + String dir = + Args.getInstance().getOutputDirectory() + Args.getInstance().getStorage().getDbDirectory(); + String enginePath = dir + File.separator + "test_engine" + File.separator + "engine.properties"; + FileUtil.createDirIfNotExists(dir + File.separator + "test_engine"); + FileUtil.createFileIfNotExists(enginePath); + PropUtil.writeProperty(enginePath, "ENGINE", "LEVELDB"); + Assert.assertEquals("LEVELDB", PropUtil.readProperty(enginePath, "ENGINE")); + + LevelDbDataSourceImpl dataSource; + dataSource = new LevelDbDataSourceImpl(dir, "test_engine"); + dataSource.closeDB(); + + PropUtil.writeProperty(enginePath, "ENGINE", "ROCKSDB"); + Assert.assertEquals("ROCKSDB", PropUtil.readProperty(enginePath, "ENGINE")); + try { + new LevelDbDataSourceImpl(dir, "test_engine"); + } catch (TronError e) { + Assert.assertEquals("Cannot open ROCKSDB database with LEVELDB engine.", e.getMessage()); + } + } + + @Test + public void testLevelDbOpenRocksDb() { + String name = "test_openRocksDb"; + String output = Paths + .get(StorageUtils.getOutputDirectoryByDbName(name), CommonParameter + .getInstance().getStorage().getDbDirectory()).toString(); + RocksDbDataSourceImpl rocksDb = new RocksDbDataSourceImpl(output, name); + rocksDb.putData(key1, value1); + rocksDb.closeDB(); + exception.expectMessage("Cannot open ROCKSDB database with LEVELDB engine."); + new LevelDbDataSourceImpl(StorageUtils.getOutputDirectoryByDbName(name), name); + } + + @Test + public void testNewInstance() { + dataSourceTest.closeDB(); + LevelDbDataSourceImpl newInst = dataSourceTest.newInstance(); + assertFalse(newInst.flush()); + newInst.closeDB(); + LevelDbDataSourceImpl empty = new LevelDbDataSourceImpl(); + empty.setDBName("empty"); + assertEquals("empty", empty.getDBName()); + String name = "newInst2"; + LevelDbDataSourceImpl newInst2 = new LevelDbDataSourceImpl( + StorageUtils.getOutputDirectoryByDbName(name), + name); + newInst2.closeDB(); + } + + @Test + public void testGetNext() { + LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( + Args.getInstance().getOutputDirectory(), "test_getNext_key"); + putSomeKeyValue(dataSource); + // case: normal + Map seekKvLimitNext = dataSource.getNext("0000000300".getBytes(), 2); + Map hashMap = Maps.newHashMap(); + hashMap.put(ByteArray.toStr(key3), ByteArray.toStr(value3)); + hashMap.put(ByteArray.toStr(key4), ByteArray.toStr(value4)); + seekKvLimitNext.forEach((key, value) -> { + String keyStr = ByteArray.toStr(key); + Assert.assertTrue("getNext", hashMap.containsKey(keyStr)); + Assert.assertEquals(ByteArray.toStr(value), hashMap.get(keyStr)); + }); + // case: targetKey greater than all existed keys + seekKvLimitNext = dataSource.getNext("0000000700".getBytes(), 2); + Assert.assertEquals(0, seekKvLimitNext.size()); + // case: limit<=0 + seekKvLimitNext = dataSource.getNext("0000000300".getBytes(), 0); + Assert.assertEquals(0, seekKvLimitNext.size()); + dataSource.closeDB(); + } + + @Test + public void testGetlatestValues() { + LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( + Args.getInstance().getOutputDirectory(), "test_getlatestValues_key"); + putSomeKeyValue(dataSource); + // case: normal + Set seekKeyLimitNext = dataSource.getlatestValues(2); + Set hashSet = Sets.newHashSet(ByteArray.toStr(value5), ByteArray.toStr(value6)); + seekKeyLimitNext.forEach(value -> { + Assert.assertTrue(hashSet.contains(ByteArray.toStr(value))); + }); + // case: limit<=0 + seekKeyLimitNext = dataSource.getlatestValues(0); + assertEquals(0, seekKeyLimitNext.size()); + dataSource.closeDB(); + } + private void makeExceptionDb(String dbName) { LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_initDb"); - dataSource.initDB(); dataSource.closeDB(); FileUtil.saveData(dataSource.getDbPath().toString() + "/CURRENT", "...", Boolean.FALSE); diff --git a/framework/src/test/java/org/tron/common/storage/leveldb/RocksDbDataSourceImplTest.java b/framework/src/test/java/org/tron/common/storage/rocksdb/RocksDbDataSourceImplTest.java similarity index 71% rename from framework/src/test/java/org/tron/common/storage/leveldb/RocksDbDataSourceImplTest.java rename to framework/src/test/java/org/tron/common/storage/rocksdb/RocksDbDataSourceImplTest.java index c6fce30e3af..86543db19fb 100644 --- a/framework/src/test/java/org/tron/common/storage/leveldb/RocksDbDataSourceImplTest.java +++ b/framework/src/test/java/org/tron/common/storage/rocksdb/RocksDbDataSourceImplTest.java @@ -1,4 +1,4 @@ -package org.tron.common.storage.leveldb; +package org.tron.common.storage.rocksdb; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertFalse; @@ -10,6 +10,8 @@ import com.google.common.collect.Sets; import java.io.File; import java.io.IOException; +import java.nio.file.Path; +import java.nio.file.Paths; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; @@ -24,13 +26,20 @@ import org.junit.Assert; import org.junit.BeforeClass; import org.junit.ClassRule; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.ExpectedException; import org.junit.rules.TemporaryFolder; -import org.tron.common.storage.rocksdb.RocksDbDataSourceImpl; +import org.rocksdb.RocksDBException; +import org.tron.common.error.TronDBException; +import org.tron.common.parameter.CommonParameter; +import org.tron.common.storage.WriteOptionsWrapper; +import org.tron.common.storage.leveldb.LevelDbDataSourceImpl; import org.tron.common.utils.ByteArray; import org.tron.common.utils.FileUtil; import org.tron.common.utils.PropUtil; import org.tron.common.utils.PublicMethod; +import org.tron.common.utils.StorageUtils; import org.tron.core.config.args.Args; import org.tron.core.db2.common.WrappedByteArray; import org.tron.core.exception.TronError; @@ -55,6 +64,9 @@ public class RocksDbDataSourceImplTest { private byte[] key5 = "00000005aa".getBytes(); private byte[] key6 = "00000006aa".getBytes(); + @Rule + public final ExpectedException expectedException = ExpectedException.none(); + /** * Release resources. */ @@ -76,7 +88,6 @@ public void testPutGet() { dataSourceTest.resetDb(); String key1 = PublicMethod.getRandomPrivateKey(); byte[] key = key1.getBytes(); - dataSourceTest.initDB(); String value1 = "50000"; byte[] value = value1.getBytes(); @@ -84,8 +95,18 @@ public void testPutGet() { assertNotNull(dataSourceTest.getData(key)); assertEquals(1, dataSourceTest.allKeys().size()); + assertEquals(1, dataSourceTest.getTotal()); + assertEquals(1, dataSourceTest.allValues().size()); assertEquals("50000", ByteArray.toStr(dataSourceTest.getData(key1.getBytes()))); + dataSourceTest.deleteData(key); + assertNull(dataSourceTest.getData(key)); + assertEquals(0, dataSourceTest.getTotal()); + dataSourceTest.iterator().forEachRemaining(entry -> Assert.fail("iterator should be empty")); + dataSourceTest.stat(); dataSourceTest.closeDB(); + dataSourceTest.stat(); // stat again + expectedException.expect(TronDBException.class); + dataSourceTest.deleteData(key); } @Test @@ -108,8 +129,6 @@ public void testReset() { public void testupdateByBatchInner() { RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_updateByBatch"); - dataSource.initDB(); - dataSource.resetDb(); String key1 = PublicMethod.getRandomPrivateKey(); String value1 = "50000"; String key2 = PublicMethod.getRandomPrivateKey(); @@ -124,6 +143,25 @@ public void testupdateByBatchInner() { assertEquals("50000", ByteArray.toStr(dataSource.getData(key1.getBytes()))); assertEquals("10000", ByteArray.toStr(dataSource.getData(key2.getBytes()))); assertEquals(2, dataSource.allKeys().size()); + + rows.clear(); + rows.put(key1.getBytes(), null); + rows.put(key2.getBytes(), null); + try (WriteOptionsWrapper options = WriteOptionsWrapper.getInstance()) { + dataSource.updateByBatch(rows, options); + } + assertEquals(0, dataSource.allKeys().size()); + + rows.clear(); + rows.put(key1.getBytes(), value1.getBytes()); + rows.put(key2.getBytes(), null); + dataSource.updateByBatch(rows); + assertEquals("50000", ByteArray.toStr(dataSource.getData(key1.getBytes()))); + assertEquals(1, dataSource.allKeys().size()); + rows.clear(); + rows.put(null, null); + expectedException.expect(RuntimeException.class); + dataSource.updateByBatch(rows); dataSource.closeDB(); } @@ -131,7 +169,6 @@ public void testupdateByBatchInner() { public void testdeleteData() { RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_delete"); - dataSource.initDB(); String key1 = PublicMethod.getRandomPrivateKey(); byte[] key = key1.getBytes(); dataSource.deleteData(key); @@ -145,8 +182,6 @@ public void testdeleteData() { public void testallKeys() { RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_find_key"); - dataSource.initDB(); - dataSource.resetDb(); String key1 = PublicMethod.getRandomPrivateKey(); byte[] key = key1.getBytes(); @@ -163,13 +198,11 @@ public void testallKeys() { dataSource.putData(key2, value2); assertEquals(2, dataSource.allKeys().size()); - dataSource.resetDb(); dataSource.closeDB(); } @Test(timeout = 1000) public void testLockReleased() { - dataSourceTest.initDB(); // normal close dataSourceTest.closeDB(); // closing already closed db. @@ -184,8 +217,6 @@ public void testLockReleased() { public void allKeysTest() { RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_allKeysTest_key"); - dataSource.initDB(); - dataSource.resetDb(); byte[] key = "0000000987b10fbb7f17110757321".getBytes(); byte[] value = "50000".getBytes(); @@ -198,7 +229,6 @@ public void allKeysTest() { logger.info(ByteArray.toStr(keyOne)); }); assertEquals(2, dataSource.allKeys().size()); - dataSource.resetDb(); dataSource.closeDB(); } @@ -224,31 +254,15 @@ private void putSomeKeyValue(RocksDbDataSourceImpl dataSource) { dataSource.putData(key4, value4); } - @Test - public void seekTest() { - RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( - Args.getInstance().getOutputDirectory(), "test_seek_key"); - dataSource.initDB(); - dataSource.resetDb(); - - putSomeKeyValue(dataSource); - Assert.assertTrue(true); - dataSource.resetDb(); - dataSource.closeDB(); - } - @Test public void getValuesNext() { RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_getValuesNext_key"); - dataSource.initDB(); - dataSource.resetDb(); putSomeKeyValue(dataSource); Set seekKeyLimitNext = dataSource.getValuesNext("0000000300".getBytes(), 2); HashSet hashSet = Sets.newHashSet(ByteArray.toStr(value3), ByteArray.toStr(value4)); seekKeyLimitNext.forEach( value -> Assert.assertTrue("getValuesNext", hashSet.contains(ByteArray.toStr(value)))); - dataSource.resetDb(); dataSource.closeDB(); } @@ -264,23 +278,17 @@ public void testCheckOrInitEngine() { RocksDbDataSourceImpl dataSource; dataSource = new RocksDbDataSourceImpl(dir, "test_engine"); - dataSource.initDB(); Assert.assertNotNull(dataSource.getDatabase()); dataSource.closeDB(); - dataSource = null; - System.gc(); PropUtil.writeProperty(enginePath, "ENGINE", "LEVELDB"); Assert.assertEquals("LEVELDB", PropUtil.readProperty(enginePath, "ENGINE")); - dataSource = new RocksDbDataSourceImpl(dir, "test_engine"); + try { - dataSource.initDB(); - } catch (Exception e) { - Assert.assertEquals(String.format("failed to check database: %s, engine do not match", - "test_engine"), - e.getMessage()); + new RocksDbDataSourceImpl(dir, "test_engine"); + } catch (TronError e) { + Assert.assertEquals("Cannot open LEVELDB database with ROCKSDB engine.", e.getMessage()); } - Assert.assertNull(dataSource.getDatabase()); PropUtil.writeProperty(enginePath, "ENGINE", "ROCKSDB"); } @@ -288,8 +296,6 @@ public void testCheckOrInitEngine() { public void testGetNext() { RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_getNext_key"); - dataSource.initDB(); - dataSource.resetDb(); putSomeKeyValue(dataSource); // case: normal Map seekKvLimitNext = dataSource.getNext("0000000300".getBytes(), 2); @@ -307,7 +313,6 @@ public void testGetNext() { // case: limit<=0 seekKvLimitNext = dataSource.getNext("0000000300".getBytes(), 0); Assert.assertEquals(0, seekKvLimitNext.size()); - dataSource.resetDb(); dataSource.closeDB(); } @@ -315,8 +320,6 @@ public void testGetNext() { public void testGetlatestValues() { RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_getlatestValues_key"); - dataSource.initDB(); - dataSource.resetDb(); putSomeKeyValue(dataSource); // case: normal Set seekKeyLimitNext = dataSource.getlatestValues(2); @@ -327,7 +330,6 @@ public void testGetlatestValues() { // case: limit<=0 seekKeyLimitNext = dataSource.getlatestValues(0); assertEquals(0, seekKeyLimitNext.size()); - dataSource.resetDb(); dataSource.closeDB(); } @@ -335,8 +337,6 @@ public void testGetlatestValues() { public void getKeysNext() { RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_getKeysNext_key"); - dataSource.initDB(); - dataSource.resetDb(); putSomeKeyValue(dataSource); int limit = 2; @@ -346,8 +346,6 @@ public void getKeysNext() { for (int i = 0; i < limit; i++) { Assert.assertArrayEquals(list.get(i), seekKeyLimitNext.get(i)); } - - dataSource.resetDb(); dataSource.closeDB(); } @@ -355,8 +353,6 @@ public void getKeysNext() { public void prefixQueryTest() { RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_prefixQuery"); - dataSource.initDB(); - dataSource.resetDb(); putSomeKeyValue(dataSource); // put a kv that will not be queried. @@ -383,23 +379,101 @@ public void prefixQueryTest() { Assert.assertEquals(list.size(), result.size()); list.forEach(entry -> Assert.assertTrue(result.contains(entry))); - dataSource.resetDb(); dataSource.closeDB(); } @Test public void initDbTest() { makeExceptionDb("test_initDb"); - RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( - Args.getInstance().getOutputDirectory(), "test_initDb"); - TronError thrown = assertThrows(TronError.class, dataSource::initDB); + TronError thrown = assertThrows(TronError.class, () -> new RocksDbDataSourceImpl( + Args.getInstance().getOutputDirectory(), "test_initDb")); assertEquals(TronError.ErrCode.ROCKSDB_INIT, thrown.getErrCode()); } + @Test + public void testRocksDbOpenLevelDb() { + String name = "test_openLevelDb"; + String output = Paths + .get(StorageUtils.getOutputDirectoryByDbName(name), CommonParameter + .getInstance().getStorage().getDbDirectory()).toString(); + LevelDbDataSourceImpl levelDb = new LevelDbDataSourceImpl( + StorageUtils.getOutputDirectoryByDbName(name), name); + levelDb.putData(key1, value1); + levelDb.closeDB(); + expectedException.expectMessage("Cannot open LEVELDB database with ROCKSDB engine."); + new RocksDbDataSourceImpl(output, name); + } + + @Test + public void testRocksDbOpenLevelDb2() { + String name = "test_openLevelDb2"; + String output = Paths + .get(StorageUtils.getOutputDirectoryByDbName(name), CommonParameter + .getInstance().getStorage().getDbDirectory()).toString(); + LevelDbDataSourceImpl levelDb = new LevelDbDataSourceImpl( + StorageUtils.getOutputDirectoryByDbName(name), name); + levelDb.putData(key1, value1); + levelDb.closeDB(); + // delete engine.properties file to simulate the case that db.engine is not set. + File engineFile = Paths.get(output, name, "engine.properties").toFile(); + if (engineFile.exists()) { + engineFile.delete(); + } + Assert.assertFalse(engineFile.exists()); + + expectedException.expectMessage("Cannot open LEVELDB database with ROCKSDB engine."); + new RocksDbDataSourceImpl(output, name); + } + + @Test + public void testNewInstance() { + dataSourceTest.closeDB(); + RocksDbDataSourceImpl newInst = dataSourceTest.newInstance(); + assertFalse(newInst.flush()); + newInst.closeDB(); + RocksDbDataSourceImpl empty = new RocksDbDataSourceImpl(); + empty.setDBName("empty"); + assertEquals("empty", empty.getDBName()); + String output = Paths + .get(StorageUtils.getOutputDirectoryByDbName("newInst2"), CommonParameter + .getInstance().getStorage().getDbDirectory()).toString(); + RocksDbDataSourceImpl newInst2 = new RocksDbDataSourceImpl(output, "newInst2"); + newInst2.closeDB(); + } + + @Test + public void backupAndDelete() throws RocksDBException { + RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( + Args.getInstance().getOutputDirectory(), "backupAndDelete"); + putSomeKeyValue(dataSource); + Path dir = Paths.get(Args.getInstance().getOutputDirectory(), "backup"); + String path = dir + File.separator; + FileUtil.createDirIfNotExists(path); + dataSource.backup(path); + File backDB = Paths.get(dir.toString(),dataSource.getDBName()).toFile(); + Assert.assertTrue(backDB.exists()); + dataSource.deleteDbBakPath(path); + Assert.assertFalse(backDB.exists()); + dataSource.closeDB(); + } + + @Test + public void testGetTotal() { + RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( + Args.getInstance().getOutputDirectory(), "test_getTotal_key"); + + Map dataMapset = Maps.newHashMap(); + dataMapset.put(key1, value1); + dataMapset.put(key2, value2); + dataMapset.put(key3, value3); + dataMapset.forEach(dataSource::putData); + Assert.assertEquals(dataMapset.size(), dataSource.getTotal()); + dataSource.closeDB(); + } + private void makeExceptionDb(String dbName) { RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "test_initDb"); - dataSource.initDB(); dataSource.closeDB(); FileUtil.saveData(dataSource.getDbPath().toString() + "/CURRENT", "...", Boolean.FALSE); diff --git a/framework/src/test/java/org/tron/common/utils/FileUtilTest.java b/framework/src/test/java/org/tron/common/utils/FileUtilTest.java index 126e0918520..c22e83760a1 100644 --- a/framework/src/test/java/org/tron/common/utils/FileUtilTest.java +++ b/framework/src/test/java/org/tron/common/utils/FileUtilTest.java @@ -67,6 +67,7 @@ public void testReadData_NormalFile() throws IOException { try (FileWriter writer = new FileWriter(tempFile.toFile())) { writer.write("Hello, World!"); } + tempFile.toFile().deleteOnExit(); char[] buffer = new char[1024]; int len = readData(tempFile.toString(), buffer); diff --git a/framework/src/test/java/org/tron/common/utils/HashCodeTest.java b/framework/src/test/java/org/tron/common/utils/HashCodeTest.java new file mode 100644 index 00000000000..36f9435c1aa --- /dev/null +++ b/framework/src/test/java/org/tron/common/utils/HashCodeTest.java @@ -0,0 +1,23 @@ +package org.tron.common.utils; + +import java.util.Objects; +import org.junit.Assert; +import org.junit.Test; +import org.tron.core.capsule.AccountCapsule; +import org.tron.core.vm.repository.Type; +import org.tron.core.vm.repository.Value; +import org.tron.protos.Protocol; + +public class HashCodeTest { + + @Test + public void test() { + Type type = new Type(); + type.setType(Type.NORMAL); + Assert.assertEquals(Integer.valueOf(Type.NORMAL).hashCode(), type.hashCode()); + Protocol.Account account = Protocol.Account.newBuilder().setBalance(100).build(); + Value value = Value.create(new AccountCapsule(account.toByteArray())); + Assert.assertEquals(Integer.valueOf( + type.hashCode() + Objects.hashCode(account)).hashCode(), value.hashCode()); + } +} diff --git a/framework/src/test/java/org/tron/common/utils/ObjectSizeUtilTest.java b/framework/src/test/java/org/tron/common/utils/ObjectSizeUtilTest.java deleted file mode 100644 index c4c72991979..00000000000 --- a/framework/src/test/java/org/tron/common/utils/ObjectSizeUtilTest.java +++ /dev/null @@ -1,62 +0,0 @@ -/* - * java-tron is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * java-tron is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.tron.common.utils; - -import static org.junit.Assert.assertEquals; - -import org.junit.Test; - -public class ObjectSizeUtilTest { - - @Test - public void testGetObjectSize() { - - Person person = new Person(); - assertEquals(48, com.carrotsearch.sizeof.RamUsageEstimator.sizeOf(person)); - Person person1 = new Person(1, "tom", new int[]{}); - assertEquals(112, com.carrotsearch.sizeof.RamUsageEstimator.sizeOf(person1)); - - Person person2 = new Person(1, "tom", new int[]{100}); - assertEquals(120, com.carrotsearch.sizeof.RamUsageEstimator.sizeOf(person2)); - - Person person3 = new Person(1, "tom", new int[]{100, 100}); - assertEquals(120, com.carrotsearch.sizeof.RamUsageEstimator.sizeOf(person3)); - Person person4 = new Person(1, "tom", new int[]{100, 100, 100}); - assertEquals(128, com.carrotsearch.sizeof.RamUsageEstimator.sizeOf(person4)); - Person person5 = new Person(1, "tom", new int[]{100, 100, 100, 100}); - assertEquals(128, com.carrotsearch.sizeof.RamUsageEstimator.sizeOf(person5)); - Person person6 = new Person(1, "tom", new int[]{100, 100, 100, 100, 100}); - assertEquals(136, com.carrotsearch.sizeof.RamUsageEstimator.sizeOf(person6)); - - } - - class Person { - - int age; - String name; - int[] scores; - - public Person() { - } - - public Person(int age, String name, int[] scores) { - this.age = age; - this.name = name; - this.scores = scores; - } - } - -} diff --git a/framework/src/test/java/org/tron/common/utils/client/Configuration.java b/framework/src/test/java/org/tron/common/utils/client/Configuration.java index 79dded303aa..fb253b9605b 100644 --- a/framework/src/test/java/org/tron/common/utils/client/Configuration.java +++ b/framework/src/test/java/org/tron/common/utils/client/Configuration.java @@ -4,7 +4,6 @@ import com.typesafe.config.ConfigFactory; import java.io.File; import java.io.FileInputStream; -import java.io.FileNotFoundException; import java.io.InputStreamReader; import org.apache.commons.lang3.StringUtils; import org.slf4j.Logger; @@ -28,12 +27,12 @@ public static Config getByPath(final String configurationPath) { if (config == null) { File configFile = new File(System.getProperty("user.dir") + '/' + configurationPath); if (configFile.exists()) { - try { - config = ConfigFactory - .parseReader(new InputStreamReader(new FileInputStream(configurationPath))); + try (FileInputStream fis = new FileInputStream(configurationPath); + InputStreamReader isr = new InputStreamReader(fis)) { + config = ConfigFactory.parseReader(isr); logger.info("use user defined config file in current dir"); - } catch (FileNotFoundException e) { - logger.error("load user defined config file exception: " + e.getMessage()); + } catch (Exception e) { + logger.error("load user defined config file exception: {}", e.getMessage()); } } else { config = ConfigFactory.load(configurationPath); diff --git a/framework/src/test/java/org/tron/common/utils/client/utils/HttpMethed.java b/framework/src/test/java/org/tron/common/utils/client/utils/HttpMethed.java index a68bb616f11..030fbd80dea 100644 --- a/framework/src/test/java/org/tron/common/utils/client/utils/HttpMethed.java +++ b/framework/src/test/java/org/tron/common/utils/client/utils/HttpMethed.java @@ -2231,7 +2231,7 @@ public static void waitToProduceOneBlock(String httpNode) { } Integer nextBlockNum = 0; Integer times = 0; - while (nextBlockNum <= currentBlockNum + 1 && times++ <= 10) { + while (nextBlockNum < currentBlockNum + 1 && times++ <= 6) { response = HttpMethed.getNowBlock(httpNode); responseContent = HttpMethed.parseResponseContent(response); if (responseContent.containsKey("block_header")) { diff --git a/framework/src/test/java/org/tron/common/utils/client/utils/TransactionUtils.java b/framework/src/test/java/org/tron/common/utils/client/utils/TransactionUtils.java index b6226d01aae..63ffe1b58ff 100644 --- a/framework/src/test/java/org/tron/common/utils/client/utils/TransactionUtils.java +++ b/framework/src/test/java/org/tron/common/utils/client/utils/TransactionUtils.java @@ -14,6 +14,7 @@ * You should have received a copy of the GNU General Public License * along with this program. If not, see . */ +import static org.tron.core.capsule.TransactionCapsule.getBase64FromByteString; import com.google.protobuf.ByteString; import java.security.SignatureException; @@ -116,20 +117,6 @@ public static byte[] getOwner(Transaction.Contract contract) { } } - /** - * constructor. - */ - - public static String getBase64FromByteString(ByteString sign) { - byte[] r = sign.substring(0, 32).toByteArray(); - byte[] s = sign.substring(32, 64).toByteArray(); - byte v = sign.byteAt(64); - if (v < 27) { - v += 27; //revId -> v - } - ECDSASignature signature = ECDSASignature.fromComponents(r, s, v); - return signature.toBase64(); - } /* * 1. check hash diff --git a/framework/src/test/java/org/tron/core/CoreExceptionTest.java b/framework/src/test/java/org/tron/core/CoreExceptionTest.java index 89feaba338c..f82b0efe326 100644 --- a/framework/src/test/java/org/tron/core/CoreExceptionTest.java +++ b/framework/src/test/java/org/tron/core/CoreExceptionTest.java @@ -29,11 +29,6 @@ import org.tron.core.exception.HeaderNotFound; import org.tron.core.exception.HighFreqException; import org.tron.core.exception.ItemNotFoundException; -import org.tron.core.exception.JsonRpcInternalException; -import org.tron.core.exception.JsonRpcInvalidParamsException; -import org.tron.core.exception.JsonRpcInvalidRequestException; -import org.tron.core.exception.JsonRpcMethodNotFoundException; -import org.tron.core.exception.JsonRpcTooManyResultException; import org.tron.core.exception.NonCommonBlockException; import org.tron.core.exception.NonUniqueObjectException; import org.tron.core.exception.P2pException; @@ -56,6 +51,11 @@ import org.tron.core.exception.ValidateSignatureException; import org.tron.core.exception.ZkProofValidateException; import org.tron.core.exception.ZksnarkException; +import org.tron.core.exception.jsonrpc.JsonRpcInternalException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidRequestException; +import org.tron.core.exception.jsonrpc.JsonRpcMethodNotFoundException; +import org.tron.core.exception.jsonrpc.JsonRpcTooManyResultException; public class CoreExceptionTest { diff --git a/framework/src/test/java/org/tron/core/CreateCommonTransactionTest.java b/framework/src/test/java/org/tron/core/CreateCommonTransactionTest.java deleted file mode 100644 index 4bcef1e148c..00000000000 --- a/framework/src/test/java/org/tron/core/CreateCommonTransactionTest.java +++ /dev/null @@ -1,47 +0,0 @@ -package org.tron.core; - -import static org.tron.common.utils.client.WalletClient.decodeFromBase58Check; - -import com.google.protobuf.Any; -import com.google.protobuf.ByteString; -import io.grpc.ManagedChannelBuilder; -import org.tron.api.GrpcAPI.TransactionExtention; -import org.tron.api.WalletGrpc; -import org.tron.api.WalletGrpc.WalletBlockingStub; -import org.tron.protos.Protocol.Transaction; -import org.tron.protos.Protocol.Transaction.Contract; -import org.tron.protos.Protocol.Transaction.Contract.ContractType; -import org.tron.protos.Protocol.Transaction.raw; -import org.tron.protos.contract.StorageContract.UpdateBrokerageContract; - -public class CreateCommonTransactionTest { - - private static final String FULL_NODE = "127.0.0.1:50051"; - - /** - * for example create UpdateBrokerageContract - */ - public static void testCreateUpdateBrokerageContract() { - WalletBlockingStub walletStub = WalletGrpc - .newBlockingStub(ManagedChannelBuilder.forTarget(FULL_NODE).usePlaintext().build()); - UpdateBrokerageContract.Builder updateBrokerageContract = UpdateBrokerageContract.newBuilder(); - updateBrokerageContract.setOwnerAddress( - ByteString.copyFrom(decodeFromBase58Check("TN3zfjYUmMFK3ZsHSsrdJoNRtGkQmZLBLz"))) - .setBrokerage(10); - Transaction.Builder transaction = Transaction.newBuilder(); - raw.Builder raw = Transaction.raw.newBuilder(); - Contract.Builder contract = Contract.newBuilder(); - contract.setType(ContractType.UpdateBrokerageContract) - .setParameter(Any.pack(updateBrokerageContract.build())); - raw.addContract(contract.build()); - transaction.setRawData(raw.build()); - TransactionExtention transactionExtention = walletStub - .createCommonTransaction(transaction.build()); - System.out.println("Common UpdateBrokerage: " + transactionExtention); - } - - public static void main(String[] args) { - testCreateUpdateBrokerageContract(); - } - -} diff --git a/framework/src/test/java/org/tron/core/ShieldWalletTest.java b/framework/src/test/java/org/tron/core/ShieldWalletTest.java index f8d5db1a44c..6e35d600ce7 100644 --- a/framework/src/test/java/org/tron/core/ShieldWalletTest.java +++ b/framework/src/test/java/org/tron/core/ShieldWalletTest.java @@ -7,6 +7,7 @@ import java.math.BigInteger; import javax.annotation.Resource; import org.junit.Assert; +import org.junit.BeforeClass; import org.junit.Test; import org.tron.api.GrpcAPI.PrivateParameters; import org.tron.api.GrpcAPI.PrivateParametersWithoutAsk; @@ -29,13 +30,14 @@ public class ShieldWalletTest extends BaseTest { @Resource private Wallet wallet; - static { - Args.setParam(new String[] {"-d", dbPath()}, Constant.TEST_CONF); + @BeforeClass + public static void init() { + Args.setParam(new String[]{"-d", dbPath()}, Constant.TEST_CONF); + librustzcashInitZksnarkParams(); } @Test public void testCreateShieldedTransaction1() { - librustzcashInitZksnarkParams(); String transactionStr1 = new String(ByteArray.fromHexString( "0x7b0a20202020227472616e73706172656e745f66726f6d5f61646472657373223a202234433930413" + "73241433344414546324536383932343545463430303839443634314345414337373433323433414233" @@ -68,7 +70,6 @@ public void testCreateShieldedTransaction1() { @Test public void testCreateShieldedTransaction2() { - librustzcashInitZksnarkParams(); String transactionStr2 = new String(ByteArray.fromHexString( "7b0a202020202261736b223a20223938666430333136376632333437623534643737323338343137663" + "6373038643537323939643938376362613838353564653037626532346236316464653064222c0a2020" @@ -176,7 +177,6 @@ public void testCreateShieldedTransaction2() { @Test public void testCreateShieldedTransactionWithoutSpendAuthSig() { - librustzcashInitZksnarkParams(); String transactionStr3 = new String(ByteArray.fromHexString( "7b0a2020202022616b223a2022373161643638633466353035373464356164333735343863626538363" + "63031663732393662393161306362303535353733313462373830383437323730326465222c0a202020" @@ -286,7 +286,6 @@ public void testCreateShieldedTransactionWithoutSpendAuthSig() { @Test public void testGetNewShieldedAddress() { - librustzcashInitZksnarkParams(); try { ShieldedAddressInfo shieldedAddressInfo = wallet.getNewShieldedAddress(); Assert.assertNotNull(shieldedAddressInfo); @@ -297,8 +296,7 @@ public void testGetNewShieldedAddress() { @Test public void testCreateShieldedContractParameters() throws ContractExeException { - librustzcashInitZksnarkParams(); - Args.getInstance().setFullNodeAllowShieldedTransactionArgs(true); + Args.getInstance().setAllowShieldedTransactionApi(true); Wallet wallet1 = spy(new Wallet()); doReturn(BigInteger.valueOf(1).toByteArray()) @@ -340,8 +338,7 @@ public void testCreateShieldedContractParameters() throws ContractExeException { @Test public void testCreateShieldedContractParameters2() throws ContractExeException { - librustzcashInitZksnarkParams(); - Args.getInstance().setFullNodeAllowShieldedTransactionArgs(true); + Args.getInstance().setAllowShieldedTransactionApi(true); Wallet wallet1 = spy(new Wallet()); doReturn(BigInteger.valueOf(1).toByteArray()) @@ -416,8 +413,7 @@ public void testCreateShieldedContractParameters2() throws ContractExeException @Test public void testCreateShieldedContractParametersWithoutAsk() throws ContractExeException { - librustzcashInitZksnarkParams(); - Args.getInstance().setFullNodeAllowShieldedTransactionArgs(true); + Args.getInstance().setAllowShieldedTransactionApi(true); Wallet wallet1 = spy(new Wallet()); doReturn(BigInteger.valueOf(1).toByteArray()) diff --git a/framework/src/test/java/org/tron/core/ShieldedTRC20BuilderTest.java b/framework/src/test/java/org/tron/core/ShieldedTRC20BuilderTest.java index 2c97473b6c3..c2c4bfe3006 100644 --- a/framework/src/test/java/org/tron/core/ShieldedTRC20BuilderTest.java +++ b/framework/src/test/java/org/tron/core/ShieldedTRC20BuilderTest.java @@ -1,7 +1,5 @@ package org.tron.core; -import static org.tron.core.zksnark.LibrustzcashTest.librustzcashInitZksnarkParams; - import com.google.protobuf.ByteString; import java.math.BigInteger; import java.util.Arrays; @@ -12,7 +10,7 @@ import org.apache.commons.lang3.tuple.Pair; import org.bouncycastle.util.encoders.Hex; import org.junit.Assert; -import org.junit.Before; +import org.junit.BeforeClass; import org.junit.Ignore; import org.junit.Test; import org.tron.api.GrpcAPI; @@ -68,7 +66,6 @@ public class ShieldedTRC20BuilderTest extends BaseTest { SHIELDED_CONTRACT_ADDRESS = WalletClient.decodeFromBase58Check(SHIELDED_CONTRACT_ADDRESS_STR); DEFAULT_OVK = ByteArray .fromHexString("030c8c2bc59fb3eb8afb047a8ea4b028743d23e7d38c6fa30908358431e2314d"); - ZksnarkInitService.librustzcashInitZksnarkParams(); PUBLIC_TO_ADDRESS = WalletClient.decodeFromBase58Check(PUBLIC_TO_ADDRESS_STR); } @@ -76,8 +73,9 @@ public class ShieldedTRC20BuilderTest extends BaseTest { VerifyTransferProof transferContract = new VerifyTransferProof(); VerifyBurnProof burnContract = new VerifyBurnProof(); - @Before - public void before() { + @BeforeClass + public static void initZksnarkParams() { + ZksnarkInitService.librustzcashInitZksnarkParams(); } @Ignore @@ -2172,7 +2170,6 @@ public void createShieldedContractParametersWithoutAskForBurn1to2() @Ignore @Test public void getTriggerInputForForMint() throws Exception { - librustzcashInitZksnarkParams(); SpendingKey sk = SpendingKey.random(); ExpandedSpendingKey expsk = sk.expandedSpendingKey(); byte[] ovk = expsk.getOvk(); @@ -2241,7 +2238,6 @@ public void getTriggerInputForForMint() throws Exception { public void testScanShieldedTRC20NotesByIvk() throws Exception { int statNum = 1; int endNum = 100; - librustzcashInitZksnarkParams(); SpendingKey sk = SpendingKey.decode(priKey); FullViewingKey fvk = sk.fullViewingKey(); byte[] ivk = fvk.inViewingKey().value; @@ -2273,7 +2269,6 @@ public void testscanShieldedTRC20NotesByOvk() throws Exception { public void isShieldedTRC20ContractNoteSpent() throws Exception { int statNum = 9200; int endNum = 9240; - librustzcashInitZksnarkParams(); SpendingKey sk = SpendingKey.decode(priKey); FullViewingKey fvk = sk.fullViewingKey(); byte[] ivk = fvk.inViewingKey().value; @@ -2350,7 +2345,6 @@ private byte[] decodePath(byte[] encodedPath) { private GrpcAPI.PrivateShieldedTRC20Parameters mintParams(String privKey, long value, String contractAddr, byte[] rcm) throws ZksnarkException, ContractValidateException { - librustzcashInitZksnarkParams(); long fromAmount = value; SpendingKey sk = SpendingKey.decode(privKey); ExpandedSpendingKey expsk = sk.expandedSpendingKey(); diff --git a/framework/src/test/java/org/tron/core/WalletMockTest.java b/framework/src/test/java/org/tron/core/WalletMockTest.java index 098ba9aee61..ab7ad7ba10c 100644 --- a/framework/src/test/java/org/tron/core/WalletMockTest.java +++ b/framework/src/test/java/org/tron/core/WalletMockTest.java @@ -835,7 +835,7 @@ public void testGetTriggerInputForShieldedTRC20Contract() { CommonParameter commonParameterMock = mock(Args.class); try (MockedStatic mockedStatic = mockStatic(CommonParameter.class)) { when(CommonParameter.getInstance()).thenReturn(commonParameterMock); - when(commonParameterMock.isFullNodeAllowShieldedTransactionArgs()).thenReturn(true); + when(commonParameterMock.isAllowShieldedTransactionApi()).thenReturn(true); assertThrows(ZksnarkException.class, () -> { wallet.getTriggerInputForShieldedTRC20Contract(triggerParam.build()); @@ -866,7 +866,7 @@ public void testGetTriggerInputForShieldedTRC20Contract1() CommonParameter commonParameterMock = mock(Args.class); try (MockedStatic mockedStatic = mockStatic(CommonParameter.class)) { when(CommonParameter.getInstance()).thenReturn(commonParameterMock); - when(commonParameterMock.isFullNodeAllowShieldedTransactionArgs()).thenReturn(true); + when(commonParameterMock.isAllowShieldedTransactionApi()).thenReturn(true); GrpcAPI.BytesMessage reponse = wallet.getTriggerInputForShieldedTRC20Contract(triggerParam.build()); @@ -1319,4 +1319,4 @@ public void testGetContractInfo1() throws Exception { wallet.getContractInfo(bytesMessage); assertNotNull(smartContractDataWrapper); } -} \ No newline at end of file +} diff --git a/framework/src/test/java/org/tron/core/WalletTest.java b/framework/src/test/java/org/tron/core/WalletTest.java index 831490fdca1..e388d3375c4 100644 --- a/framework/src/test/java/org/tron/core/WalletTest.java +++ b/framework/src/test/java/org/tron/core/WalletTest.java @@ -29,6 +29,8 @@ import com.google.protobuf.Any; import com.google.protobuf.ByteString; + +import java.util.ArrayList; import java.util.Arrays; import javax.annotation.Resource; import lombok.SneakyThrows; @@ -66,9 +68,13 @@ import org.tron.core.capsule.TransactionCapsule; import org.tron.core.capsule.TransactionInfoCapsule; import org.tron.core.capsule.TransactionResultCapsule; +import org.tron.core.capsule.VotesCapsule; +import org.tron.core.capsule.WitnessCapsule; import org.tron.core.config.args.Args; +import org.tron.core.db2.core.Chainbase; import org.tron.core.exception.ContractExeException; import org.tron.core.exception.ContractValidateException; +import org.tron.core.exception.MaintenanceUnavailableException; import org.tron.core.exception.NonUniqueObjectException; import org.tron.core.store.DynamicPropertiesStore; import org.tron.core.utils.ProposalUtil.ProposalType; @@ -851,6 +857,153 @@ public void testGetDelegatedResourceV2() { } } + @Test + public void testGetPaginatedNowWitnessList_Error() { + try { + // To avoid throw MaintenanceClearingException + dbManager.getChainBaseManager().getDynamicPropertiesStore().saveStateFlag(1); + wallet.getPaginatedNowWitnessList(0, 10); + Assert.fail("Should throw error when in maintenance period"); + } catch (Exception e) { + Assert.assertTrue("Should throw MaintenanceClearingException", + e instanceof MaintenanceUnavailableException); + } + + try { + Args.getInstance().setSolidityNode(true); + wallet.getPaginatedNowWitnessList(0, 10); + Args.getInstance().setSolidityNode(false); + + dbManager.setCursor(Chainbase.Cursor.SOLIDITY); + wallet.getPaginatedNowWitnessList(0, 10); + dbManager.setCursor(Chainbase.Cursor.HEAD); + } catch (Exception e) { + Assert.assertFalse("Should not throw MaintenanceClearingException", + e instanceof MaintenanceUnavailableException); + } + + dbManager.getChainBaseManager().getDynamicPropertiesStore().saveStateFlag(0); + } + + @Test + public void testGetPaginatedNowWitnessList_CornerCase() { + try { + // To avoid throw MaintenanceClearingException + dbManager.getChainBaseManager().getDynamicPropertiesStore().saveStateFlag(0); + + GrpcAPI.WitnessList witnessList = wallet.getPaginatedNowWitnessList(-100, 0); + Assert.assertTrue("Should return an empty witness list when offset is negative", + witnessList == null); + + witnessList = wallet.getPaginatedNowWitnessList(100, 0); + Assert.assertTrue("Should return an empty witness list when limit is 0", witnessList == null); + + String fakeWitnessAddressPrefix = "fake_witness"; + int fakeNumberOfWitnesses = 1000 + 10; + // Mock additional witnesses with vote counts greater than 1000 + for (int i = 0; i < fakeNumberOfWitnesses; i++) { + saveWitnessWith(fakeWitnessAddressPrefix + i, 200); + } + + witnessList = wallet.getPaginatedNowWitnessList(0, 1000000); + // Check the returned witness list should contain 1000 witnesses with descending vote count + Assert.assertTrue("Witness list should contain 1000 witnesses", + witnessList.getWitnessesCount() == 1000); + + // clean up, delete the fake witnesses + for (int i = 0; i < fakeNumberOfWitnesses; i++) { + chainBaseManager.getWitnessStore() + .delete(ByteString.copyFromUtf8(fakeWitnessAddressPrefix + i).toByteArray()); + } + } catch (MaintenanceUnavailableException e) { + Assert.fail(e.getMessage()); + } + } + + @Test + public void testGetPaginatedNowWitnessList() { + GrpcAPI.WitnessList witnessList = wallet.getWitnessList(); + logger.info(witnessList.toString()); + + // iterate through the witness list and find the existing maximum vote count + long maxVoteCount = 0L; + for (Protocol.Witness witness : witnessList.getWitnessesList()) { + if (witness.getVoteCount() > maxVoteCount) { + maxVoteCount = witness.getVoteCount(); + } + } + String fakeWitnessAddressPrefix = "fake_witness_address_for_paged_now_witness_list"; + int fakeNumberOfWitnesses = 10; + // Mock additional witnesses with vote counts greater than the maximum + for (int i = 0; i < fakeNumberOfWitnesses; i++) { + saveWitnessWith(fakeWitnessAddressPrefix + i, maxVoteCount + 1000000L); + } + + // Create a VotesCapsule to simulate the votes for the fake witnesses + VotesCapsule votesCapsule = new VotesCapsule(ByteString.copyFromUtf8(ACCOUNT_ADDRESS_ONE), + new ArrayList()); + votesCapsule.addOldVotes(ByteString.copyFromUtf8(fakeWitnessAddressPrefix + 0), 100L); + votesCapsule.addOldVotes(ByteString.copyFromUtf8(fakeWitnessAddressPrefix + 1), 50L); + votesCapsule.addNewVotes(ByteString.copyFromUtf8(fakeWitnessAddressPrefix + 2), 200L); + votesCapsule.addNewVotes(ByteString.copyFromUtf8(fakeWitnessAddressPrefix + 3), 300L); + chainBaseManager.getVotesStore().put(votesCapsule.createDbKey(), votesCapsule); + + logger.info("now request paginated witness list with 0 offset and 10 limit:"); + GrpcAPI.WitnessList witnessList2 = null; + try { + // To avoid throw MaintenanceClearingException + dbManager.getChainBaseManager().getDynamicPropertiesStore().saveStateFlag(0); + witnessList2 = wallet.getPaginatedNowWitnessList(0, 10); + } catch (MaintenanceUnavailableException e) { + Assert.fail(e.getMessage()); + } + // Check the returned witness list should contain 10 witnesses with descending vote count + Assert.assertTrue("Witness list should contain 10 witnesses", + witnessList2.getWitnessesCount() == 10); + // Check the first witness should have the maximum vote count + Assert.assertEquals("The first witness should have the maximum vote count", + fakeWitnessAddressPrefix + 3, witnessList2.getWitnesses(0).getAddress().toStringUtf8()); + Assert.assertEquals(maxVoteCount + 1000300L, witnessList2.getWitnesses(0).getVoteCount()); + // Check the second witness should have the second maximum vote count + Assert.assertEquals("The second witness", fakeWitnessAddressPrefix + 2, + witnessList2.getWitnesses(1).getAddress().toStringUtf8()); + Assert.assertEquals(maxVoteCount + 1000200L, witnessList2.getWitnesses(1).getVoteCount()); + // Check the last witness should have the least vote count + Assert.assertEquals("The tenth witness", fakeWitnessAddressPrefix + 0, + witnessList2.getWitnesses(9).getAddress().toStringUtf8()); + Assert.assertEquals(maxVoteCount + 1000000L - 100L, + witnessList2.getWitnesses(9).getVoteCount()); + + + logger.info("after paged"); + GrpcAPI.WitnessList witnessList3 = wallet.getWitnessList(); + // Check the witness list should remain unchanged after paged request + for (Protocol.Witness witness : witnessList3.getWitnessesList()) { + if (witness.getVoteCount() > maxVoteCount) { + Assert.assertTrue("Check the witness list should remain unchanged after paged request", + witness.getVoteCount() == maxVoteCount + 1000000L); + } + } + + // clean up, delete the fake witnesses + for (int i = 0; i < fakeNumberOfWitnesses; i++) { + chainBaseManager.getWitnessStore() + .delete(ByteString.copyFromUtf8(fakeWitnessAddressPrefix + i).toByteArray()); + } + chainBaseManager.getVotesStore().delete(votesCapsule.createDbKey()); + Assert.assertTrue("Clean up the mocked witness data", + wallet.getWitnessList().getWitnessesCount() == witnessList.getWitnessesCount()); + + } + + public void saveWitnessWith(String witnessAddress, long voteCount) { + WitnessCapsule witness = new WitnessCapsule( + Protocol.Witness.newBuilder() + .setAddress(ByteString.copyFromUtf8(witnessAddress)) // Convert String to ByteString + .setVoteCount(voteCount).build()); + chainBaseManager.getWitnessStore().put(witness.getAddress().toByteArray(), witness); + } + @Test public void testGetDelegatedResourceAccountIndexV2() { dbManager.getDynamicPropertiesStore().saveUnfreezeDelayDays(1L); diff --git a/framework/src/test/java/org/tron/core/actuator/AssetIssueActuatorTest.java b/framework/src/test/java/org/tron/core/actuator/AssetIssueActuatorTest.java index 77c24c28797..7daf139dc0f 100755 --- a/framework/src/test/java/org/tron/core/actuator/AssetIssueActuatorTest.java +++ b/framework/src/test/java/org/tron/core/actuator/AssetIssueActuatorTest.java @@ -1,6 +1,7 @@ package org.tron.core.actuator; import static org.junit.Assert.fail; +import static org.tron.core.config.Parameter.ChainConstant.FROZEN_PERIOD; import com.google.protobuf.Any; import com.google.protobuf.ByteString; @@ -15,12 +16,14 @@ import org.junit.Test; import org.tron.common.BaseTest; import org.tron.common.utils.ByteArray; +import org.tron.common.utils.ForkController; import org.tron.core.Constant; import org.tron.core.Wallet; import org.tron.core.capsule.AccountCapsule; import org.tron.core.capsule.AssetIssueCapsule; import org.tron.core.capsule.TransactionResultCapsule; import org.tron.core.config.Parameter.ForkBlockVersionConsts; +import org.tron.core.config.Parameter.ForkBlockVersionEnum; import org.tron.core.config.args.Args; import org.tron.core.exception.ContractExeException; import org.tron.core.exception.ContractValidateException; @@ -1868,6 +1871,53 @@ public void SameTokenNameCloseInvalidAccount() { } + @Test + public void issueStartTimeTooBig() { + long maintenanceTimeInterval = dbManager.getDynamicPropertiesStore() + .getMaintenanceTimeInterval(); + long hardForkTime = + ((ForkBlockVersionEnum.VERSION_4_0_1.getHardForkTime() - 1) / maintenanceTimeInterval + 1) + * maintenanceTimeInterval; + dbManager.getDynamicPropertiesStore() + .saveLatestBlockHeaderTimestamp(hardForkTime + 1); + + // add more check after 4.8.1 + byte[] stats = new byte[27]; + Arrays.fill(stats, (byte) 1); + dbManager.getDynamicPropertiesStore() + .statsByVersion(ForkBlockVersionEnum.VERSION_4_8_1.getValue(), stats); + boolean flag = ForkController.instance().pass(ForkBlockVersionEnum.VERSION_4_8_1); + Assert.assertTrue(flag); + + TransactionResultCapsule ret = new TransactionResultCapsule(); + + // Start time is too big. If it's to large, the account.frozen_supply.expireTime will overflow + FrozenSupply frozenSupply = FrozenSupply.newBuilder().setFrozenDays(20).setFrozenAmount(100) + .build(); + long startTime = Long.MAX_VALUE - frozenSupply.getFrozenDays() * FROZEN_PERIOD + 1; + Any any = Any.pack( + AssetIssueContract.newBuilder() + .setOwnerAddress(ByteString.copyFrom(ByteArray.fromHexString(OWNER_ADDRESS))) + .setName(ByteString.copyFromUtf8(NAME)).setTotalSupply(TOTAL_SUPPLY).setTrxNum(TRX_NUM) + .setNum(NUM) + .setStartTime(startTime) + .setEndTime(startTime + 24 * 3600 * 1000) + .setDescription(ByteString.copyFromUtf8(DESCRIPTION)) + .setUrl(ByteString.copyFromUtf8(URL)) + .setPrecision(3) + .addFrozenSupply(frozenSupply) + .build()); + AssetIssueActuator actuator = new AssetIssueActuator(); + actuator.setChainBaseManager(dbManager.getChainBaseManager()).setAny(any); + + AccountCapsule owner = dbManager.getAccountStore().get(ByteArray.fromHexString(OWNER_ADDRESS)); + owner.setBalance(10_000_000_000L); + dbManager.getAccountStore().put(owner.createDbKey(), owner); + + processAndCheckInvalid(actuator, ret, + "Start time and frozen days would cause expire time overflow", + "Start time and frozen days would cause expire time overflow"); + } @Test public void commonErrorCheck() { diff --git a/framework/src/test/java/org/tron/core/actuator/ExchangeTransactionActuatorTest.java b/framework/src/test/java/org/tron/core/actuator/ExchangeTransactionActuatorTest.java index d39706e0699..fbce246101e 100644 --- a/framework/src/test/java/org/tron/core/actuator/ExchangeTransactionActuatorTest.java +++ b/framework/src/test/java/org/tron/core/actuator/ExchangeTransactionActuatorTest.java @@ -1,10 +1,13 @@ package org.tron.core.actuator; +import static org.junit.Assert.assertThrows; import static org.junit.Assert.fail; import static org.tron.core.config.Parameter.ChainSymbol.TRX_SYMBOL_BYTES; import com.google.protobuf.Any; import com.google.protobuf.ByteString; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; import java.util.Arrays; import java.util.Map; import junit.framework.TestCase; @@ -13,18 +16,30 @@ import org.junit.Before; import org.junit.Test; import org.tron.common.BaseTest; +import org.tron.common.crypto.ECKey; import org.tron.common.utils.ByteArray; +import org.tron.common.utils.ForkController; +import org.tron.common.utils.PublicMethod; +import org.tron.consensus.base.Param; +import org.tron.consensus.base.Param.Miner; import org.tron.core.Constant; import org.tron.core.Wallet; import org.tron.core.capsule.AccountCapsule; import org.tron.core.capsule.AssetIssueCapsule; +import org.tron.core.capsule.BlockCapsule; import org.tron.core.capsule.ExchangeCapsule; +import org.tron.core.capsule.TransactionCapsule; import org.tron.core.capsule.TransactionResultCapsule; +import org.tron.core.capsule.WitnessCapsule; +import org.tron.core.config.Parameter; +import org.tron.core.config.Parameter.ForkBlockVersionEnum; import org.tron.core.config.args.Args; +import org.tron.core.db.Manager; import org.tron.core.exception.ContractExeException; import org.tron.core.exception.ContractValidateException; import org.tron.core.exception.ItemNotFoundException; import org.tron.protos.Protocol.AccountType; +import org.tron.protos.Protocol.Transaction.Contract.ContractType; import org.tron.protos.Protocol.Transaction.Result.code; import org.tron.protos.contract.AssetIssueContractOuterClass; import org.tron.protos.contract.AssetIssueContractOuterClass.AssetIssueContract; @@ -1674,7 +1689,7 @@ public void noContract() { public void invalidContractType() { ExchangeTransactionActuator actuator = new ExchangeTransactionActuator(); // create AssetIssueContract, not a valid ClearABI contract , which will throw e expectipon - Any invalidContractTypes = Any.pack(AssetIssueContractOuterClass.AssetIssueContract.newBuilder() + Any invalidContractTypes = Any.pack(AssetIssueContract.newBuilder() .build()); actuator.setChainBaseManager(dbManager.getChainBaseManager()) .setAny(invalidContractTypes); @@ -1725,4 +1740,92 @@ private void processAndCheckInvalid(ExchangeTransactionActuator actuator, } } + /** + * isExchangeTransaction + */ + @Test + public void isExchangeTransactionPush() { + try { + TransactionCapsule transactionCap = new TransactionCapsule( + ExchangeTransactionContract.newBuilder() + .setOwnerAddress(ByteString.copyFrom(ByteArray.fromHexString(OWNER_ADDRESS_SECOND))) + .setExchangeId(1) + .setTokenId(ByteString.copyFrom("_".getBytes())) + .setQuant(1) + .setExpected(1) + .build(), ContractType.ExchangeTransactionContract); + dbManager.pushTransaction(transactionCap); + + } catch (Exception e) { + Assert.assertTrue(true); + } + } + + @Test + public void isExchangeTransactionGenerate() { + try { + + String key = PublicMethod.getRandomPrivateKey(); + byte[] privateKey = ByteArray.fromHexString(key); + final ECKey ecKey = ECKey.fromPrivate(privateKey); + byte[] address = ecKey.getAddress(); + WitnessCapsule witnessCapsule = new WitnessCapsule(ByteString.copyFrom(address)); + + String OWNER_ADDRESS_SECOND = + Wallet.getAddressPreFixString() + "548794500882809695a8a687866e76d4271a1abc"; + TransactionCapsule transactionCap = new TransactionCapsule( + ExchangeTransactionContract.newBuilder() + .setOwnerAddress(ByteString.copyFrom(ByteArray.fromHexString(OWNER_ADDRESS_SECOND))) + .setExchangeId(1) + .setTokenId(ByteString.copyFrom("_".getBytes())) + .setQuant(1) + .setExpected(1) + .build(), ContractType.ExchangeTransactionContract); + dbManager.getPendingTransactions().add(transactionCap); + Param param = Param.getInstance(); + Miner miner = param.new Miner(privateKey, witnessCapsule.getAddress(), + witnessCapsule.getAddress()); + BlockCapsule blockCapsule = dbManager + .generateBlock(miner, 1533529947843L, System.currentTimeMillis() + 1000); + } catch (Exception e) { + Assert.assertTrue(false); + } + } + + @Test + public void rejectExchangeTransaction() { + try { + long maintenanceTimeInterval = dbManager.getDynamicPropertiesStore() + .getMaintenanceTimeInterval(); + long hardForkTime = + ((ForkBlockVersionEnum.VERSION_4_0_1.getHardForkTime() - 1) / maintenanceTimeInterval + 1) + * maintenanceTimeInterval; + dbManager.getDynamicPropertiesStore() + .saveLatestBlockHeaderTimestamp(hardForkTime + 1); + byte[] stats = new byte[27]; + Arrays.fill(stats, (byte) 1); + dbManager.getDynamicPropertiesStore() + .statsByVersion(ForkBlockVersionEnum.VERSION_4_8_0_1.getValue(), stats); + boolean flag = ForkController.instance().pass(ForkBlockVersionEnum.VERSION_4_8_0_1); + Assert.assertTrue(flag); + String OWNER_ADDRESS_SECOND = + Wallet.getAddressPreFixString() + "548794500882809695a8a687866e76d4271a1abc"; + TransactionCapsule transactionCap = new TransactionCapsule( + ExchangeTransactionContract.newBuilder() + .setOwnerAddress(ByteString.copyFrom(ByteArray.fromHexString(OWNER_ADDRESS_SECOND))) + .setExchangeId(1) + .setTokenId(ByteString.copyFrom("_".getBytes())) + .setQuant(1) + .setExpected(1) + .build(), ContractType.ExchangeTransactionContract); + Method rejectExchangeTransaction = Manager.class.getDeclaredMethod( + "rejectExchangeTransaction", org.tron.protos.Protocol.Transaction.class); + rejectExchangeTransaction.setAccessible(true); + Exception ex = assertThrows(InvocationTargetException.class, () -> { + rejectExchangeTransaction.invoke(dbManager, transactionCap.getInstance()); + }); + } catch (Exception e) { + fail(); + } + } } diff --git a/framework/src/test/java/org/tron/core/actuator/ShieldedTransferActuatorTest.java b/framework/src/test/java/org/tron/core/actuator/ShieldedTransferActuatorTest.java index b71ba432018..cb95194f3d3 100755 --- a/framework/src/test/java/org/tron/core/actuator/ShieldedTransferActuatorTest.java +++ b/framework/src/test/java/org/tron/core/actuator/ShieldedTransferActuatorTest.java @@ -85,7 +85,7 @@ public class ShieldedTransferActuatorTest extends BaseTest { */ @BeforeClass public static void init() throws ZksnarkException { - Args.setFullNodeAllowShieldedTransaction(true); + Args.getInstance().setAllowShieldedTransactionApi(true); librustzcashInitZksnarkParams(); } @@ -950,7 +950,7 @@ public void publicAddressAToShieldAddressNoToAddressFailure() { */ @Test public void publicToShieldAddressAndShieldToPublicAddressWithZoreValueSuccess() { - Args.setFullNodeAllowShieldedTransaction(true); + Args.getInstance().setAllowShieldedTransactionApi(true); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); long fee = dbManager.getDynamicPropertiesStore().getShieldedTransactionFee(); diff --git a/framework/src/test/java/org/tron/core/actuator/utils/ProposalUtilTest.java b/framework/src/test/java/org/tron/core/actuator/utils/ProposalUtilTest.java index e8a1e862f54..2ca466fb4da 100644 --- a/framework/src/test/java/org/tron/core/actuator/utils/ProposalUtilTest.java +++ b/framework/src/test/java/org/tron/core/actuator/utils/ProposalUtilTest.java @@ -1,5 +1,8 @@ package org.tron.core.actuator.utils; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertThrows; + import com.google.protobuf.ByteString; import java.util.ArrayList; import java.util.Arrays; @@ -11,6 +14,7 @@ import org.junit.Assert; import org.junit.BeforeClass; import org.junit.Test; +import org.junit.function.ThrowingRunnable; import org.tron.common.BaseTest; import org.tron.common.utils.ByteArray; import org.tron.common.utils.ForkController; @@ -439,6 +443,10 @@ public void validateCheck() { testAllowTvmBlobProposal(); + testAllowMarketTransaction(); + + testAllowTvmSelfdestructRestrictionProposal(); + forkUtils.getManager().getDynamicPropertiesStore() .statsByVersion(ForkBlockVersionEnum.ENERGY_LIMIT.getValue(), stats); forkUtils.reset(); @@ -659,6 +667,102 @@ private void testAllowTvmBlobProposal() { } + private void testAllowTvmSelfdestructRestrictionProposal() { + byte[] stats = new byte[27]; + forkUtils.getManager().getDynamicPropertiesStore() + .statsByVersion(ForkBlockVersionEnum.VERSION_4_8_1.getValue(), stats); + try { + ProposalUtil.validator(dynamicPropertiesStore, forkUtils, + ProposalType.ALLOW_TVM_SELFDESTRUCT_RESTRICTION.getCode(), 1); + Assert.fail(); + } catch (ContractValidateException e) { + Assert.assertEquals( + "Bad chain parameter id [ALLOW_TVM_SELFDESTRUCT_RESTRICTION]", + e.getMessage()); + } + + long maintenanceTimeInterval = forkUtils.getManager().getDynamicPropertiesStore() + .getMaintenanceTimeInterval(); + + long hardForkTime = + ((ForkBlockVersionEnum.VERSION_4_8_1.getHardForkTime() - 1) / maintenanceTimeInterval + 1) + * maintenanceTimeInterval; + forkUtils.getManager().getDynamicPropertiesStore() + .saveLatestBlockHeaderTimestamp(hardForkTime + 1); + + stats = new byte[27]; + Arrays.fill(stats, (byte) 1); + forkUtils.getManager().getDynamicPropertiesStore() + .statsByVersion(ForkBlockVersionEnum.VERSION_4_8_1.getValue(), stats); + + // Should fail because the proposal value is invalid + try { + ProposalUtil.validator(dynamicPropertiesStore, forkUtils, + ProposalType.ALLOW_TVM_SELFDESTRUCT_RESTRICTION.getCode(), 2); + Assert.fail(); + } catch (ContractValidateException e) { + Assert.assertEquals( + "This value[ALLOW_TVM_SELFDESTRUCT_RESTRICTION] is only allowed to be 1", + e.getMessage()); + } + + dynamicPropertiesStore.saveAllowTvmSelfdestructRestriction(1); + try { + ProposalUtil.validator(dynamicPropertiesStore, forkUtils, + ProposalType.ALLOW_TVM_SELFDESTRUCT_RESTRICTION.getCode(), 1); + Assert.fail(); + } catch (ContractValidateException e) { + Assert.assertEquals( + "[ALLOW_TVM_SELFDESTRUCT_RESTRICTION] has been valid, no need to propose again", + e.getMessage()); + } + } + + private void testAllowMarketTransaction() { + ThrowingRunnable off = () -> ProposalUtil.validator(dynamicPropertiesStore, forkUtils, + ProposalType.ALLOW_MARKET_TRANSACTION.getCode(), 0); + ThrowingRunnable open = () -> ProposalUtil.validator(dynamicPropertiesStore, forkUtils, + ProposalType.ALLOW_MARKET_TRANSACTION.getCode(), 1); + String err = "Bad chain parameter id [ALLOW_MARKET_TRANSACTION]"; + + ContractValidateException thrown = assertThrows(ContractValidateException.class, open); + assertEquals(err, thrown.getMessage()); + + activateFork(ForkBlockVersionEnum.VERSION_4_1); + + try { + open.run(); + } catch (Throwable e) { + Assert.fail(e.getMessage()); + } + + thrown = assertThrows(ContractValidateException.class, off); + assertEquals("This value[ALLOW_MARKET_TRANSACTION] is only allowed to be 1", + thrown.getMessage()); + + activateFork(ForkBlockVersionEnum.VERSION_4_8_1); + + thrown = assertThrows(ContractValidateException.class, open); + assertEquals(err, thrown.getMessage()); + + thrown = assertThrows(ContractValidateException.class, off); + assertEquals(err, thrown.getMessage()); + } + + private void activateFork(ForkBlockVersionEnum forkVersion) { + byte[] stats = new byte[27]; + Arrays.fill(stats, (byte) 1); + forkUtils.getManager().getDynamicPropertiesStore() + .statsByVersion(forkVersion.getValue(), stats); + + long maintenanceTimeInterval = forkUtils.getManager().getDynamicPropertiesStore() + .getMaintenanceTimeInterval(); + long hardForkTime = ((forkVersion.getHardForkTime() - 1) / maintenanceTimeInterval + 1) + * maintenanceTimeInterval; + forkUtils.getManager().getDynamicPropertiesStore() + .saveLatestBlockHeaderTimestamp(hardForkTime + 1); + } + @Test public void blockVersionCheck() { for (ForkBlockVersionEnum forkVersion : ForkBlockVersionEnum.values()) { diff --git a/framework/src/test/java/org/tron/core/actuator/vm/ProgramTraceListenerTest.java b/framework/src/test/java/org/tron/core/actuator/vm/ProgramTraceListenerTest.java index 089711219f8..770e2bd0ea5 100644 --- a/framework/src/test/java/org/tron/core/actuator/vm/ProgramTraceListenerTest.java +++ b/framework/src/test/java/org/tron/core/actuator/vm/ProgramTraceListenerTest.java @@ -2,6 +2,7 @@ import java.io.IOException; import java.lang.reflect.Field; +import java.util.ArrayList; import java.util.List; import java.util.Map; import lombok.extern.slf4j.Slf4j; @@ -15,12 +16,15 @@ import org.tron.core.Constant; import org.tron.core.config.args.Args; import org.tron.core.db.TransactionStoreTest; +import org.tron.core.vm.trace.Op; import org.tron.core.vm.trace.OpActions; import org.tron.core.vm.trace.OpActions.Action; +import org.tron.core.vm.trace.ProgramTrace; import org.tron.core.vm.trace.ProgramTraceListener; @Slf4j(topic = "VM") public class ProgramTraceListenerTest { + @ClassRule public static TemporaryFolder temporaryFolder = new TemporaryFolder(); @@ -33,7 +37,7 @@ public class ProgramTraceListenerTest { @BeforeClass public static void init() throws IOException { - Args.setParam(new String[]{"--output-directory", + Args.setParam(new String[] {"--output-directory", temporaryFolder.newFolder().toString(), "--debug"}, Constant.TEST_CONF); } @@ -130,7 +134,6 @@ private void validateProgramTraceListener() { Assert.assertFalse(e instanceof IllegalAccessException); } - traceListener.resetActions(); try { @@ -179,4 +182,25 @@ public void programTraceListenerTest() { validateDisableTraceListener(); } + @Test + public void testGetSet() { + ProgramTrace programTrace = new ProgramTrace(); + Op op = new Op(); + List ops = new ArrayList<>(); + ops.add(op); + programTrace.setOps(ops); + programTrace.setResult("result"); + programTrace.setContractAddress("contractAddress"); + programTrace.setError("error"); + programTrace.result(new byte[] {}); + programTrace.error(new Exception()); + programTrace.getOps(); + programTrace.getContractAddress(); + programTrace.getError(); + programTrace.getResult(); + programTrace.toString(); + + Assert.assertTrue(true); + } + } diff --git a/framework/src/test/java/org/tron/core/actuator/vm/SerializersTest.java b/framework/src/test/java/org/tron/core/actuator/vm/SerializersTest.java new file mode 100644 index 00000000000..52ee1eeb937 --- /dev/null +++ b/framework/src/test/java/org/tron/core/actuator/vm/SerializersTest.java @@ -0,0 +1,12 @@ +package org.tron.core.actuator.vm; + +import org.junit.Test; +import org.tron.core.vm.trace.Serializers; + +public class SerializersTest { + + @Test + public void testSerializeFieldsOnly() { + Serializers.serializeFieldsOnly("testString", true); + } +} diff --git a/framework/src/test/java/org/tron/core/capsule/BlockCapsuleTest.java b/framework/src/test/java/org/tron/core/capsule/BlockCapsuleTest.java index 3c86d893895..552a014a97b 100644 --- a/framework/src/test/java/org/tron/core/capsule/BlockCapsuleTest.java +++ b/framework/src/test/java/org/tron/core/capsule/BlockCapsuleTest.java @@ -134,7 +134,7 @@ public void testHasWitnessSignature() { localWitnesses = new LocalWitnesses(); localWitnesses.setPrivateKeys(Arrays.asList(privateKey)); - localWitnesses.initWitnessAccountAddress(true); + localWitnesses.initWitnessAccountAddress(null, true); Args.setLocalWitnesses(localWitnesses); Assert.assertFalse(blockCapsule0.hasWitnessSignature()); diff --git a/framework/src/test/java/org/tron/core/capsule/utils/ExchangeProcessorTest.java b/framework/src/test/java/org/tron/core/capsule/utils/ExchangeProcessorTest.java index 717c62b01a8..1f0be4b1f7c 100644 --- a/framework/src/test/java/org/tron/core/capsule/utils/ExchangeProcessorTest.java +++ b/framework/src/test/java/org/tron/core/capsule/utils/ExchangeProcessorTest.java @@ -138,12 +138,63 @@ public void testWithdraw() { @Test public void testStrictMath() { long supply = 1_000_000_000_000_000_000L; - ExchangeProcessor processor = new ExchangeProcessor(supply, false); - long anotherTokenQuant = processor.exchange(4732214, 2202692725330L, 29218); - processor = new ExchangeProcessor(supply, true); - long result = processor.exchange(4732214, 2202692725330L, 29218); - Assert.assertNotEquals(anotherTokenQuant, result); + long[][] testData = { + {4732214L, 2202692725330L, 29218L}, + {5618633L, 556559904655L, 1L}, + {9299554L, 1120271441185L, 7000L}, + {62433133L, 12013267997895L, 100000L}, + {64212664L, 725836766395L, 50000L}, + {64126212L, 2895100109660L, 5000L}, + {56459055L, 3288380567368L, 165000L}, + {21084707L, 1589204008960L, 50000L}, + {24120521L, 1243764649177L, 20000L}, + {836877L, 212532333234L, 5293L}, + {55879741L, 13424854054078L, 250000L}, + {66388882L, 11300012790454L, 300000L}, + {94470955L, 7941038150919L, 2000L}, + {13613746L, 5012660712983L, 122L}, + {71852829L, 5262251868618L, 396L}, + {3857658L, 446109245044L, 20637L}, + {35491863L, 3887393269796L, 100L}, + {295632118L, 1265298439004L, 500000L}, + {49320113L, 1692106302503L, 123267L}, + {10966984L, 6222910652894L, 2018L}, + {41634280L, 2004508994767L, 865L}, + {10087714L, 6765558834714L, 1009L}, + {42270078L, 210360843525L, 200000L}, + {571091915L, 655011397250L, 2032520L}, + {51026781L, 1635726339365L, 37L}, + {61594L, 312318864132L, 500L}, + {11616684L, 5875978057357L, 20L}, + {60584529L, 1377717821301L, 78132L}, + {29818073L, 3033545989651L, 182L}, + {3855280L, 834647482043L, 16L}, + {58310711L, 1431562205655L, 200000L}, + {60226263L, 1386036785882L, 178226L}, + {3537634L, 965771433992L, 225L}, + {3760534L, 908700758784L, 328L}, + {80913L, 301864126445L, 4L}, + {3789271L, 901842209723L, 1L}, + {4051904L, 843419481286L, 1005L}, + {89141L, 282107742510L, 100L}, + {90170L, 282854635378L, 26L}, + {4229852L, 787503315944L, 137L}, + {4259884L, 781975090197L, 295L}, + {3627657L, 918682223700L, 34L}, + {813519L, 457546358759L, 173L}, + {89626L, 327856173057L, 27L}, + {97368L, 306386489550L, 50L}, + {93712L, 305866015731L, 4L}, + {3281260L, 723656594544L, 40L}, + {3442652L, 689908773685L, 18L}, + }; + + for (long[] data : testData) { + ExchangeProcessor processor = new ExchangeProcessor(supply, false); + long anotherTokenQuant = processor.exchange(data[0], data[1], data[2]); + processor = new ExchangeProcessor(supply, true); + long result = processor.exchange(data[0], data[1], data[2]); + Assert.assertNotEquals(anotherTokenQuant, result); + } } - - } diff --git a/framework/src/test/java/org/tron/core/config/args/ArgsTest.java b/framework/src/test/java/org/tron/core/config/args/ArgsTest.java index 4bb8e7e4909..fb19528b626 100644 --- a/framework/src/test/java/org/tron/core/config/args/ArgsTest.java +++ b/framework/src/test/java/org/tron/core/config/args/ArgsTest.java @@ -57,7 +57,8 @@ public void destroy() { @Test public void get() { - Args.setParam(new String[] {"-c", Constant.TEST_CONF}, Constant.TESTNET_CONF); + Args.setParam(new String[] {"-c", Constant.TEST_CONF, "--keystore-factory"}, + Constant.TESTNET_CONF); CommonParameter parameter = Args.getInstance(); @@ -65,10 +66,10 @@ public void get() { localWitnesses = new LocalWitnesses(); localWitnesses.setPrivateKeys(Arrays.asList(privateKey)); - localWitnesses.initWitnessAccountAddress(true); + localWitnesses.initWitnessAccountAddress(null, true); Args.setLocalWitnesses(localWitnesses); address = ByteArray.toHexString(Args.getLocalWitnesses() - .getWitnessAccountAddress(CommonParameter.getInstance().isECKeyCryptoEngine())); + .getWitnessAccountAddress()); Assert.assertEquals(Constant.ADD_PRE_FIX_STRING_TESTNET, DecodeUtil.addressPreFixString); Assert.assertEquals(0, parameter.getBackupPriority()); @@ -126,7 +127,9 @@ public void get() { Assert.assertEquals(address, ByteArray.toHexString(Args.getLocalWitnesses() - .getWitnessAccountAddress(CommonParameter.getInstance().isECKeyCryptoEngine()))); + .getWitnessAccountAddress())); + + Assert.assertTrue(parameter.isKeystoreFactory()); } @Test diff --git a/framework/src/test/java/org/tron/core/config/args/LocalWitnessTest.java b/framework/src/test/java/org/tron/core/config/args/LocalWitnessTest.java index 27d5effd6b1..7f6d5417924 100644 --- a/framework/src/test/java/org/tron/core/config/args/LocalWitnessTest.java +++ b/framework/src/test/java/org/tron/core/config/args/LocalWitnessTest.java @@ -15,24 +15,47 @@ package org.tron.core.config.args; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertThrows; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + import com.google.common.collect.Lists; +import java.io.IOException; +import java.security.SecureRandom; +import org.junit.AfterClass; import org.junit.Assert; import org.junit.Before; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.tron.common.utils.ByteArray; import org.tron.common.utils.LocalWitnesses; import org.tron.common.utils.PublicMethod; +import org.tron.common.utils.StringUtil; +import org.tron.core.Constant; +import org.tron.core.exception.TronError; +import org.tron.core.exception.TronError.ErrCode; public class LocalWitnessTest { private final LocalWitnesses localWitness = new LocalWitnesses(); private static final String PRIVATE_KEY = PublicMethod.getRandomPrivateKey(); + @Rule + public final TemporaryFolder temporaryFolder = new TemporaryFolder(); + @Before public void setLocalWitness() { localWitness .setPrivateKeys( Lists.newArrayList( - PRIVATE_KEY)); + PRIVATE_KEY)); + } + + @AfterClass + public static void clear() { + Args.clearParam(); } @Test @@ -42,16 +65,16 @@ public void whenSetNullPrivateKey() { Assert.assertNotNull(localWitness.getPublicKey()); } - @Test + @Test(expected = TronError.class) public void whenSetEmptyPrivateKey() { localWitness.setPrivateKeys(Lists.newArrayList("")); - Assert.assertNotNull(localWitness.getPrivateKey()); - Assert.assertNotNull(localWitness.getPublicKey()); + fail("private key must be 64-bits hex string"); } - @Test(expected = IllegalArgumentException.class) + @Test(expected = TronError.class) public void whenSetBadFormatPrivateKey() { localWitness.setPrivateKeys(Lists.newArrayList("a111")); + fail("private key must be 64-bits hex string"); } @Test @@ -65,6 +88,68 @@ public void whenSetPrefixPrivateKey() { Assert.assertNotNull(localWitness.getPrivateKey()); } + @Test + public void testValidPrivateKey() { + LocalWitnesses localWitnesses = new LocalWitnesses(); + + try { + localWitnesses.addPrivateKeys(PRIVATE_KEY); + Assert.assertEquals(1, localWitnesses.getPrivateKeys().size()); + Assert.assertEquals(PRIVATE_KEY, localWitnesses.getPrivateKeys().get(0)); + } catch (Exception e) { + fail(e.getMessage()); + } + } + + @Test + public void testValidPrivateKeyWithPrefix() { + LocalWitnesses localWitnesses = new LocalWitnesses(); + + try { + localWitnesses.addPrivateKeys("0x" + PRIVATE_KEY); + Assert.assertEquals(1, localWitnesses.getPrivateKeys().size()); + Assert.assertEquals("0x" + PRIVATE_KEY, localWitnesses.getPrivateKeys().get(0)); + } catch (Exception e) { + fail(e.getMessage()); + } + } + + @Test + public void testInvalidPrivateKey() { + LocalWitnesses localWitnesses = new LocalWitnesses(); + String expectedMessage = "private key must be 64 hex string"; + assertTronError(localWitnesses, null, expectedMessage); + assertTronError(localWitnesses, "", expectedMessage); + assertTronError(localWitnesses, " ", expectedMessage); + assertTronError(localWitnesses, "11111", expectedMessage); + String expectedMessage2 = "private key must be hex string"; + SecureRandom secureRandom = new SecureRandom(); + byte[] keyBytes = new byte[31]; + secureRandom.nextBytes(keyBytes); + final String privateKey = ByteArray.toHexString(keyBytes) + " "; + assertTronError(localWitnesses, privateKey, expectedMessage2); + final String privateKey2 = "xy" + ByteArray.toHexString(keyBytes); + assertTronError(localWitnesses, privateKey2, expectedMessage2); + } + + private void assertTronError(LocalWitnesses localWitnesses, String privateKey, + String expectedMessage) { + TronError thrown = assertThrows(TronError.class, + () -> localWitnesses.addPrivateKeys(privateKey)); + assertEquals(ErrCode.WITNESS_INIT, thrown.getErrCode()); + assertTrue(thrown.getMessage().contains(expectedMessage)); + } + + @Test + public void testHexStringFormat() { + Assert.assertTrue(StringUtil.isHexadecimal("0123456789abcdefABCDEF")); + Assert.assertFalse(StringUtil.isHexadecimal(null)); + Assert.assertFalse(StringUtil.isHexadecimal("")); + Assert.assertFalse(StringUtil.isHexadecimal("abc")); + Assert.assertFalse(StringUtil.isHexadecimal(" ")); + Assert.assertFalse(StringUtil.isHexadecimal("123xyz")); + } + @Test public void getPrivateKey() { Assert.assertEquals(Lists @@ -77,14 +162,34 @@ public void testConstructor() { LocalWitnesses localWitnesses = new LocalWitnesses(PublicMethod.getRandomPrivateKey()); LocalWitnesses localWitnesses1 = new LocalWitnesses(Lists.newArrayList(PublicMethod.getRandomPrivateKey())); - localWitnesses.setWitnessAccountAddress(new byte[0]); + localWitnesses.initWitnessAccountAddress(new byte[0], true); Assert.assertNotNull(localWitnesses1.getPublicKey()); LocalWitnesses localWitnesses2 = new LocalWitnesses(); Assert.assertNull(localWitnesses2.getPrivateKey()); Assert.assertNull(localWitnesses2.getPublicKey()); - localWitnesses2.initWitnessAccountAddress(true); + localWitnesses2.initWitnessAccountAddress(null, true); LocalWitnesses localWitnesses3 = new LocalWitnesses(); - Assert.assertNotNull(localWitnesses3.getWitnessAccountAddress(true)); + Assert.assertNull(localWitnesses3.getWitnessAccountAddress()); + } + + @Test + public void testLocalWitnessConfig() throws IOException { + Args.setParam( + new String[]{"--output-directory", temporaryFolder.newFolder().toString(), "-w", "--debug"}, + "config-localtest.conf"); + LocalWitnesses witness = Args.getLocalWitnesses(); + Assert.assertNotNull(witness.getPrivateKey()); + Assert.assertNotNull(witness.getWitnessAccountAddress()); + } + + @Test + public void testNullLocalWitnessConfig() throws IOException { + Args.setParam( + new String[]{"--output-directory", temporaryFolder.newFolder().toString(), "--debug"}, + Constant.TEST_CONF); + LocalWitnesses witness = Args.getLocalWitnesses(); + Assert.assertNull(witness.getPrivateKey()); + Assert.assertNull(witness.getWitnessAccountAddress()); } } diff --git a/framework/src/test/java/org/tron/core/config/args/WitnessInitializerTest.java b/framework/src/test/java/org/tron/core/config/args/WitnessInitializerTest.java new file mode 100644 index 00000000000..7364b1f9b3a --- /dev/null +++ b/framework/src/test/java/org/tron/core/config/args/WitnessInitializerTest.java @@ -0,0 +1,268 @@ +package org.tron.core.config.args; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertThrows; +import static org.junit.Assert.assertTrue; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyString; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.mockStatic; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import com.typesafe.config.Config; +import com.typesafe.config.ConfigFactory; +import java.io.File; +import java.lang.reflect.Field; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; +import java.util.ArrayList; +import java.util.List; +import org.bouncycastle.util.encoders.Hex; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.mockito.MockedStatic; +import org.tron.common.crypto.SignInterface; +import org.tron.common.parameter.CommonParameter; +import org.tron.common.utils.ByteArray; +import org.tron.common.utils.LocalWitnesses; +import org.tron.common.utils.PublicMethod; +import org.tron.common.utils.client.utils.Base58; +import org.tron.core.Constant; +import org.tron.core.exception.TronError; +import org.tron.core.exception.TronError.ErrCode; +import org.tron.keystore.Credentials; +import org.tron.keystore.WalletUtils; + +public class WitnessInitializerTest { + + private Config config; + private WitnessInitializer witnessInitializer; + + private static final String privateKey = PublicMethod.getRandomPrivateKey(); + private static final String address = Base58.encode58Check( + ByteArray.fromHexString(PublicMethod.getHexAddressByPrivateKey(privateKey))); + private static final String invalidAddress = "RJCzdnv88Hvqa2jB1C9dMmMYHr5DFdF2R3"; + + @Before + public void setUp() { + config = ConfigFactory.empty(); + witnessInitializer = new WitnessInitializer(config); + } + + @After + public void clear() { + Args.clearParam(); + } + + @Test + public void testInitLocalWitnessesEmpty() { + Args.PARAMETER.setWitness(false); + + LocalWitnesses result = witnessInitializer.initLocalWitnesses(); + assertNotNull(result); + assertTrue(result.getPrivateKeys().isEmpty()); + + Args.PARAMETER.setWitness(true); + LocalWitnesses localWitnesses = witnessInitializer.initLocalWitnesses(); + assertTrue(localWitnesses.getPrivateKeys().isEmpty()); + + String configString = "localwitness = [] \n localwitnesskeystore = []"; + config = ConfigFactory.parseString(configString); + witnessInitializer = new WitnessInitializer(config); + localWitnesses = witnessInitializer.initLocalWitnesses(); + assertTrue(localWitnesses.getPrivateKeys().isEmpty()); + } + + @Test + public void testTryInitFromCommandLine() + throws NoSuchFieldException, IllegalAccessException, NoSuchMethodException, + InvocationTargetException { + Field privateKeyField = CommonParameter.class.getDeclaredField("privateKey"); + privateKeyField.setAccessible(true); + privateKeyField.set(Args.getInstance(), ""); + + witnessInitializer = new WitnessInitializer(config); + Method method = WitnessInitializer.class.getDeclaredMethod( + "tryInitFromCommandLine"); + method.setAccessible(true); + boolean result = (boolean) method.invoke(witnessInitializer); + assertFalse(result); + + privateKeyField.set(Args.getInstance(), privateKey); + method.invoke(witnessInitializer); + result = (boolean) method.invoke(witnessInitializer); + assertTrue(result); + + Field witnessAddress = CommonParameter.class.getDeclaredField("witnessAddress"); + witnessAddress.setAccessible(true); + witnessAddress.set(Args.getInstance(), address); + result = (boolean) method.invoke(witnessInitializer); + assertTrue(result); + + witnessAddress.set(Args.getInstance(), invalidAddress); + InvocationTargetException thrown = assertThrows(InvocationTargetException.class, + () -> method.invoke(witnessInitializer)); + TronError targetException = (TronError) thrown.getTargetException(); + assertEquals(ErrCode.WITNESS_INIT, targetException.getErrCode()); + } + + @Test + public void testTryInitFromConfig() + throws NoSuchMethodException, InvocationTargetException, IllegalAccessException { + witnessInitializer = new WitnessInitializer(config); + Method method = WitnessInitializer.class.getDeclaredMethod( + "tryInitFromConfig"); + method.setAccessible(true); + boolean result = (boolean) method.invoke(witnessInitializer); + assertFalse(result); + + String configString = "localwitness = []"; + config = ConfigFactory.parseString(configString); + witnessInitializer = new WitnessInitializer(config); + result = (boolean) method.invoke(witnessInitializer); + assertFalse(result); + + configString = "localwitness = [" + privateKey + "]"; + config = ConfigFactory.parseString(configString); + witnessInitializer = new WitnessInitializer(config); + result = (boolean) method.invoke(witnessInitializer); + assertTrue(result); + + configString = "localWitnessAccountAddress = " + address + "\n" + + "localwitness = [\n" + privateKey + "]"; + config = ConfigFactory.parseString(configString); + witnessInitializer = new WitnessInitializer(config); + result = (boolean) method.invoke(witnessInitializer); + assertTrue(result); + + configString = "localwitness = [\n" + privateKey + "\n" + privateKey + "]"; + config = ConfigFactory.parseString(configString); + witnessInitializer = new WitnessInitializer(config); + result = (boolean) method.invoke(witnessInitializer); + assertTrue(result); + + configString = "localWitnessAccountAddress = " + invalidAddress + "\n" + + "localwitness = [\n" + privateKey + "]"; + config = ConfigFactory.parseString(configString); + witnessInitializer = new WitnessInitializer(config); + InvocationTargetException thrown = assertThrows(InvocationTargetException.class, + () -> method.invoke(witnessInitializer)); + TronError targetException = (TronError) thrown.getTargetException(); + assertEquals(ErrCode.WITNESS_INIT, targetException.getErrCode()); + + configString = "localWitnessAccountAddress = " + address + "\n" + + "localwitness = [\n" + privateKey + "\n" + privateKey + "]"; + config = ConfigFactory.parseString(configString); + witnessInitializer = new WitnessInitializer(config); + thrown = assertThrows(InvocationTargetException.class, + () -> method.invoke(witnessInitializer)); + targetException = (TronError) thrown.getTargetException(); + assertEquals(ErrCode.WITNESS_INIT, targetException.getErrCode()); + } + + @Test + public void testTryInitFromKeystore() + throws NoSuchMethodException, InvocationTargetException, IllegalAccessException, + NoSuchFieldException { + witnessInitializer = new WitnessInitializer(config); + Method method = WitnessInitializer.class.getDeclaredMethod( + "tryInitFromKeystore"); + method.setAccessible(true); + method.invoke(witnessInitializer); + Field localWitnessField = WitnessInitializer.class.getDeclaredField("localWitnesses"); + localWitnessField.setAccessible(true); + LocalWitnesses localWitnesses = (LocalWitnesses) localWitnessField.get(witnessInitializer); + assertTrue(localWitnesses.getPrivateKeys().isEmpty()); + + String configString = "localwitnesskeystore = []"; + Config emptyListConfig = ConfigFactory.parseString(configString); + witnessInitializer = new WitnessInitializer(emptyListConfig); + method.invoke(witnessInitializer); + localWitnesses = (LocalWitnesses) localWitnessField.get(witnessInitializer); + assertTrue(localWitnesses.getPrivateKeys().isEmpty()); + } + + @Test + public void testTryInitFromKeyStore2() + throws NoSuchFieldException, IllegalAccessException { + Args.PARAMETER.setWitness(true); + Config mockConfig = mock(Config.class); + when(mockConfig.hasPath(Constant.LOCAL_WITNESS_KEYSTORE)).thenReturn(false); + witnessInitializer = new WitnessInitializer(mockConfig); + witnessInitializer.initLocalWitnesses(); + verify(mockConfig, never()).getStringList(anyString()); + + when(mockConfig.hasPath(Constant.LOCAL_WITNESS_KEYSTORE)).thenReturn(true); + when(mockConfig.getStringList(Constant.LOCAL_WITNESS_KEYSTORE)).thenReturn(new ArrayList<>()); + witnessInitializer = new WitnessInitializer(mockConfig); + witnessInitializer.initLocalWitnesses(); + verify(mockConfig, times(1)).getStringList(Constant.LOCAL_WITNESS_KEYSTORE); + + List keystores = new ArrayList<>(); + keystores.add("keystore1.json"); + keystores.add("keystore2.json"); + when(mockConfig.hasPath(Constant.LOCAL_WITNESS_KEYSTORE)).thenReturn(true); + when(mockConfig.getStringList(Constant.LOCAL_WITNESS_KEYSTORE)).thenReturn(keystores); + + Field password = CommonParameter.class.getDeclaredField("password"); + password.setAccessible(true); + password.set(Args.getInstance(), "password"); + + try (MockedStatic mockedWalletUtils = mockStatic(WalletUtils.class); + MockedStatic mockedByteArray = mockStatic(ByteArray.class)) { + // Mock WalletUtils.loadCredentials + Credentials credentials = mock(Credentials.class); + SignInterface signInterface = mock(SignInterface.class); + when(credentials.getSignInterface()).thenReturn(signInterface); + byte[] keyBytes = Hex.decode(privateKey); + when(signInterface.getPrivateKey()).thenReturn(keyBytes); + mockedWalletUtils.when(() -> WalletUtils.loadCredentials(anyString(), any(File.class))) + .thenReturn(credentials); + mockedByteArray.when(() -> ByteArray.toHexString(any())).thenReturn(privateKey); + mockedByteArray.when(() -> ByteArray.fromHexString(anyString())).thenReturn(keyBytes); + + witnessInitializer = new WitnessInitializer(mockConfig); + Field localWitnessField = WitnessInitializer.class.getDeclaredField("localWitnesses"); + localWitnessField.setAccessible(true); + localWitnessField.set(witnessInitializer, new LocalWitnesses(privateKey)); + LocalWitnesses localWitnesses = witnessInitializer.initLocalWitnesses(); + assertFalse(localWitnesses.getPrivateKeys().isEmpty()); + } + } + + @Test + public void testGetWitnessAddress() + throws InvocationTargetException, IllegalAccessException, NoSuchMethodException, + NoSuchFieldException { + witnessInitializer = new WitnessInitializer(config); + Method method = WitnessInitializer.class.getDeclaredMethod( + "getWitnessAddress"); + method.setAccessible(true); + byte[] result = (byte[]) method.invoke(witnessInitializer); + assertNull(result); + + String configString = "localWitnessAccountAddress = " + address; + config = ConfigFactory.parseString(configString); + witnessInitializer = new WitnessInitializer(config); + Field localWitnessField = WitnessInitializer.class.getDeclaredField("localWitnesses"); + localWitnessField.setAccessible(true); + localWitnessField.set(witnessInitializer, new LocalWitnesses(privateKey)); + result = (byte[]) method.invoke(witnessInitializer); + assertNotNull(result); + + configString = "localWitnessAccountAddress = " + invalidAddress; + config = ConfigFactory.parseString(configString); + witnessInitializer = new WitnessInitializer(config); + InvocationTargetException thrown = assertThrows(InvocationTargetException.class, + () -> method.invoke(witnessInitializer)); + TronError targetException = (TronError) thrown.getTargetException(); + assertEquals(ErrCode.WITNESS_INIT, targetException.getErrCode()); + } +} diff --git a/framework/src/test/java/org/tron/core/db/AccountStoreTest.java b/framework/src/test/java/org/tron/core/db/AccountStoreTest.java index 9249a3358dc..aab64df16c7 100755 --- a/framework/src/test/java/org/tron/core/db/AccountStoreTest.java +++ b/framework/src/test/java/org/tron/core/db/AccountStoreTest.java @@ -1,20 +1,29 @@ package org.tron.core.db; import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.mock; import com.google.protobuf.ByteString; +import com.typesafe.config.Config; +import java.lang.reflect.Field; +import java.util.ArrayList; +import java.util.Collections; import java.util.HashMap; +import java.util.List; import java.util.Map; import javax.annotation.Resource; import org.junit.Assert; import org.junit.Before; import org.junit.Test; +import org.mockito.Mockito; import org.tron.common.BaseTest; import org.tron.common.utils.ByteArray; import org.tron.core.Constant; import org.tron.core.capsule.AccountCapsule; import org.tron.core.config.args.Args; import org.tron.core.db2.ISession; +import org.tron.core.exception.TronError; +import org.tron.core.net.peer.PeerManager; import org.tron.core.store.AccountStore; import org.tron.core.store.AssetIssueStore; import org.tron.core.store.DynamicPropertiesStore; @@ -61,6 +70,21 @@ public void init() { init = true; } + @Test + public void setAccountTest() throws Exception { + Field field = AccountStore.class.getDeclaredField("assertsAddress"); + field.setAccessible(true); + field.set(AccountStore.class, new HashMap<>()); + Config config = mock(Config.class); + Mockito.when(config.getObjectList("genesis.block.assets")).thenReturn(new ArrayList<>()); + try { + AccountStore.setAccount(config); + Assert.fail(); + } catch (Throwable e) { + Assert.assertTrue(e instanceof TronError); + } + } + @Test public void get() { //test get and has Method diff --git a/framework/src/test/java/org/tron/core/db/DBIteratorTest.java b/framework/src/test/java/org/tron/core/db/DBIteratorTest.java index b4f7ca424c0..100502428d0 100644 --- a/framework/src/test/java/org/tron/core/db/DBIteratorTest.java +++ b/framework/src/test/java/org/tron/core/db/DBIteratorTest.java @@ -14,6 +14,7 @@ import org.junit.Test; import org.junit.rules.ExpectedException; import org.junit.rules.TemporaryFolder; +import org.rocksdb.ReadOptions; import org.rocksdb.RocksDB; import org.rocksdb.RocksDBException; import org.tron.core.db.common.iterator.RockStoreIterator; @@ -83,7 +84,7 @@ public void testRocksDb() throws RocksDBException, IOException { RocksDB db = RocksDB.open(options, file.toString())) { db.put("1".getBytes(StandardCharsets.UTF_8), "1".getBytes(StandardCharsets.UTF_8)); db.put("2".getBytes(StandardCharsets.UTF_8), "2".getBytes(StandardCharsets.UTF_8)); - RockStoreIterator iterator = new RockStoreIterator(db.newIterator()); + RockStoreIterator iterator = new RockStoreIterator(db.newIterator(), new ReadOptions()); iterator.seekToFirst(); Assert.assertArrayEquals("1".getBytes(StandardCharsets.UTF_8), iterator.getKey()); Assert.assertArrayEquals("1".getBytes(StandardCharsets.UTF_8), iterator.next().getValue()); @@ -99,7 +100,7 @@ public void testRocksDb() throws RocksDBException, IOException { Assert.assertTrue(e instanceof IllegalStateException); } - iterator = new RockStoreIterator(db.newIterator()); + iterator = new RockStoreIterator(db.newIterator(), new ReadOptions()); iterator.seekToLast(); Assert.assertArrayEquals("2".getBytes(StandardCharsets.UTF_8), iterator.getKey()); Assert.assertArrayEquals("2".getBytes(StandardCharsets.UTF_8), iterator.getValue()); diff --git a/framework/src/test/java/org/tron/core/db/ManagerTest.java b/framework/src/test/java/org/tron/core/db/ManagerTest.java index db219377b74..fc3b5c18265 100755 --- a/framework/src/test/java/org/tron/core/db/ManagerTest.java +++ b/framework/src/test/java/org/tron/core/db/ManagerTest.java @@ -33,6 +33,7 @@ import org.junit.Rule; import org.junit.Test; import org.junit.rules.TemporaryFolder; +import org.tron.api.GrpcAPI; import org.tron.common.application.TronApplicationContext; import org.tron.common.crypto.ECKey; import org.tron.common.runtime.RuntimeImpl; @@ -52,7 +53,10 @@ import org.tron.core.capsule.AccountCapsule; import org.tron.core.capsule.AssetIssueCapsule; import org.tron.core.capsule.BlockCapsule; +import org.tron.core.capsule.BytesCapsule; import org.tron.core.capsule.TransactionCapsule; +import org.tron.core.capsule.TransactionInfoCapsule; +import org.tron.core.capsule.TransactionRetCapsule; import org.tron.core.capsule.WitnessCapsule; import org.tron.core.config.DefaultConfig; import org.tron.core.config.Parameter; @@ -136,7 +140,7 @@ public void init() throws IOException { localWitnesses = new LocalWitnesses(); localWitnesses.setPrivateKeys(Arrays.asList(privateKey)); - localWitnesses.initWitnessAccountAddress(true); + localWitnesses.initWitnessAccountAddress(null, true); Args.setLocalWitnesses(localWitnesses); blockCapsule2 = @@ -316,7 +320,7 @@ public void transactionTest() { dbManager.pushTransaction(trans0); dbManager.pushTransaction(trans); } catch (Exception e) { - Assert.assertTrue(e instanceof TaposException); + Assert.assertTrue(e instanceof ContractValidateException); } dbManager.rePush(trans0); ReflectUtils.invokeMethod(dbManager,"filterOwnerAddress", @@ -858,6 +862,35 @@ public void getVerifyTxsTest() { dbManager.getPendingTransactions().add(t3); txs = dbManager.getVerifyTxs(capsule); Assert.assertEquals(txs.size(), 2); + + dbManager.getPendingTransactions().clear(); + capsule = new BlockCapsule(0, ByteString.EMPTY, 0, list); + dbManager.getPendingTransactions().add(t1); + dbManager.getPendingTransactions().add(t2); + txs = dbManager.getVerifyTxs(capsule); + Assert.assertEquals(txs.size(), 0); + + dbManager.getPendingTransactions().clear(); + Transaction t1Bak = t1.getInstance().toBuilder() + .addSignature(ByteString.copyFrom("a".getBytes())).build(); + dbManager.getPendingTransactions().add(new TransactionCapsule(t1Bak)); + txs = dbManager.getVerifyTxs(capsule); + Assert.assertEquals(t1.getTransactionId(), new TransactionCapsule(t1Bak).getTransactionId()); + Assert.assertEquals(txs.size(), 2); + + dbManager.getPendingTransactions().clear(); + list.clear(); + list.add(t1Bak); + capsule = new BlockCapsule(0, ByteString.EMPTY, 0, list); + + Transaction t2Bak = t1.getInstance().toBuilder() + .addSignature(ByteString.copyFrom("a".getBytes())) + .addSignature(ByteString.copyFrom("b".getBytes())).build(); + Assert.assertEquals(new TransactionCapsule(t1Bak).getTransactionId(), + new TransactionCapsule(t2Bak).getTransactionId()); + dbManager.getPendingTransactions().add(new TransactionCapsule(t2Bak)); + txs = dbManager.getVerifyTxs(capsule); + Assert.assertEquals(txs.size(), 1); } @Test @@ -1233,6 +1266,47 @@ public void testExpiration() { } + @Test + public void testGetTransactionInfoByBlockNum() throws Exception { + + Transaction transaction = Protocol.Transaction.newBuilder() + .addSignature(ByteString.copyFrom(new byte[1])).build(); + TransactionCapsule transactionCapsule = new TransactionCapsule(transaction); + + Protocol.BlockHeader.raw raw = Protocol.BlockHeader.raw.newBuilder().setNumber(1000L).build(); + Protocol.BlockHeader header = Protocol.BlockHeader.newBuilder().setRawData(raw).build(); + Block block = Block.newBuilder().setBlockHeader(header).addTransactions(transaction).build(); + + Protocol.TransactionInfo info = Protocol.TransactionInfo.newBuilder() + .setBlockNumber(1000L).build(); + + BlockCapsule blockCapsule = new BlockCapsule(block); + byte[] blockId = new BlockCapsule(block).getBlockId().getBytes(); + dbManager.getBlockIndexStore().put(ByteArray.fromLong(1000L), new BytesCapsule(blockId)); + dbManager.getBlockStore().put(blockId, blockCapsule); + dbManager.getTransactionHistoryStore().put(transactionCapsule.getTransactionId().getBytes(), + new TransactionInfoCapsule(info)); + + GrpcAPI.TransactionInfoList transactionInfoList = dbManager.getTransactionInfoByBlockNum(1000L); + + Assert.assertEquals(1, transactionInfoList.getTransactionInfoCount()); + Assert.assertEquals(1, transactionInfoList.getTransactionInfoList().size()); + + Protocol.TransactionRet ret = Protocol.TransactionRet.newBuilder() + .addTransactioninfo(info) + .addTransactioninfo(info).build(); + + TransactionRetCapsule transactionRetCapsule = new TransactionRetCapsule(ret.toByteArray()); + + dbManager.getTransactionRetStore() + .put(ByteArray.fromLong(1000L), transactionRetCapsule); + + transactionInfoList = dbManager.getTransactionInfoByBlockNum(1000L); + + Assert.assertEquals(2, transactionInfoList.getTransactionInfoCount()); + Assert.assertEquals(2, transactionInfoList.getTransactionInfoList().size()); + } + @Test public void blockTrigger() { Manager manager = spy(new Manager()); diff --git a/framework/src/test/java/org/tron/core/db/TransactionExpireTest.java b/framework/src/test/java/org/tron/core/db/TransactionExpireTest.java index 420b54525e4..f563203b71a 100644 --- a/framework/src/test/java/org/tron/core/db/TransactionExpireTest.java +++ b/framework/src/test/java/org/tron/core/db/TransactionExpireTest.java @@ -57,7 +57,7 @@ private void initLocalWitness() { String randomPrivateKey = PublicMethod.getRandomPrivateKey(); LocalWitnesses localWitnesses = new LocalWitnesses(); localWitnesses.setPrivateKeys(Arrays.asList(randomPrivateKey)); - localWitnesses.initWitnessAccountAddress(true); + localWitnesses.initWitnessAccountAddress(null, true); Args.setLocalWitnesses(localWitnesses); } @@ -85,7 +85,7 @@ public void testExpireTransaction() { TransferContract transferContract = TransferContract.newBuilder() .setAmount(1L) .setOwnerAddress(ByteString.copyFrom(Args.getLocalWitnesses() - .getWitnessAccountAddress(CommonParameter.getInstance().isECKeyCryptoEngine()))) + .getWitnessAccountAddress())) .setToAddress(ByteString.copyFrom(ByteArray.fromHexString( (Wallet.getAddressPreFixString() + "A389132D6639FBDA4FBC8B659264E6B7C90DB086")))) .build(); @@ -116,8 +116,7 @@ public void testExpireTransactionNew() { .saveLatestBlockHeaderTimestamp(blockCapsule.getTimeStamp()); dbManager.updateRecentBlock(blockCapsule); initLocalWitness(); - byte[] address = Args.getLocalWitnesses() - .getWitnessAccountAddress(CommonParameter.getInstance().isECKeyCryptoEngine()); + byte[] address = Args.getLocalWitnesses().getWitnessAccountAddress(); ByteString addressByte = ByteString.copyFrom(address); AccountCapsule accountCapsule = new AccountCapsule(Protocol.Account.newBuilder().setAddress(addressByte).build()); @@ -157,8 +156,7 @@ public void testTransactionApprovedList() { dbManager.updateRecentBlock(blockCapsule); initLocalWitness(); - byte[] address = Args.getLocalWitnesses() - .getWitnessAccountAddress(CommonParameter.getInstance().isECKeyCryptoEngine()); + byte[] address = Args.getLocalWitnesses().getWitnessAccountAddress(); TransferContract transferContract = TransferContract.newBuilder() .setAmount(1L) .setOwnerAddress(ByteString.copyFrom(address)) diff --git a/framework/src/test/java/org/tron/core/db/WitnessStoreTest.java b/framework/src/test/java/org/tron/core/db/WitnessStoreTest.java index d141a5fd790..521d048f23e 100755 --- a/framework/src/test/java/org/tron/core/db/WitnessStoreTest.java +++ b/framework/src/test/java/org/tron/core/db/WitnessStoreTest.java @@ -62,12 +62,12 @@ public void testSortWitness() { List witnessAddress = allWitnesses.stream().map(WitnessCapsule::getAddress) .collect(Collectors.toList()); this.witnessStore.sortWitness(witnessAddress, false); - this.witnessStore.sortWitnesses(allWitnesses, false); + WitnessStore.sortWitnesses(allWitnesses, false); Assert.assertEquals(witnessAddress, allWitnesses.stream().map(WitnessCapsule::getAddress) .collect(Collectors.toList())); List pre = new ArrayList<>(witnessAddress); this.witnessStore.sortWitness(witnessAddress, true); - this.witnessStore.sortWitnesses(allWitnesses, true); + WitnessStore.sortWitnesses(allWitnesses, true); Assert.assertEquals(witnessAddress, allWitnesses.stream().map(WitnessCapsule::getAddress) .collect(Collectors.toList())); Assert.assertNotEquals(pre, witnessAddress); diff --git a/framework/src/test/java/org/tron/core/db2/ChainbaseTest.java b/framework/src/test/java/org/tron/core/db2/ChainbaseTest.java index ee9813be633..4ab36ad2c88 100644 --- a/framework/src/test/java/org/tron/core/db2/ChainbaseTest.java +++ b/framework/src/test/java/org/tron/core/db2/ChainbaseTest.java @@ -76,9 +76,7 @@ public void initDb() throws IOException { public void testPrefixQueryForLeveldb() { LevelDbDataSourceImpl dataSource = new LevelDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "testPrefixQueryForLeveldb"); - dataSource.initDB(); - this.chainbase = new Chainbase(new SnapshotRoot( - new LevelDB(dataSource))); + this.chainbase = new Chainbase(new SnapshotRoot(new LevelDB(dataSource))); testDb(chainbase); testRoot(dataSource); chainbase.reset(); @@ -89,7 +87,6 @@ public void testPrefixQueryForLeveldb() { public void testPrefixQueryForRocksdb() { RocksDbDataSourceImpl dataSource = new RocksDbDataSourceImpl( Args.getInstance().getOutputDirectory(), "testPrefixQueryForRocksdb"); - dataSource.initDB(); this.chainbase = new Chainbase(new SnapshotRoot( new org.tron.core.db2.common.RocksDB(dataSource))); testDb(chainbase); diff --git a/framework/src/test/java/org/tron/core/db2/CheckpointV2Test.java b/framework/src/test/java/org/tron/core/db2/CheckpointV2Test.java index dff2d376fd5..fb7f1987e9f 100644 --- a/framework/src/test/java/org/tron/core/db2/CheckpointV2Test.java +++ b/framework/src/test/java/org/tron/core/db2/CheckpointV2Test.java @@ -4,7 +4,7 @@ import com.google.common.primitives.Bytes; import com.google.common.primitives.Longs; import com.google.protobuf.ByteString; -import java.io.File; +import java.io.IOException; import java.util.Iterator; import java.util.List; import java.util.Map; @@ -13,11 +13,12 @@ import org.junit.After; import org.junit.Assert; import org.junit.Before; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.TemporaryFolder; import org.tron.common.application.Application; import org.tron.common.application.ApplicationFactory; import org.tron.common.application.TronApplicationContext; -import org.tron.common.utils.FileUtil; import org.tron.common.utils.Sha256Hash; import org.tron.core.Constant; import org.tron.core.capsule.BlockCapsule; @@ -34,10 +35,12 @@ public class CheckpointV2Test { private TronApplicationContext context; private Application appT; private TestRevokingTronStore tronDatabase; + @Rule + public final TemporaryFolder temporaryFolder = new TemporaryFolder(); @Before - public void init() { - Args.setParam(new String[]{"-d", "output_SnapshotManager_test"}, + public void init() throws IOException { + Args.setParam(new String[]{"-d", temporaryFolder.newFolder().toString()}, Constant.TEST_CONF); Args.getInstance().getStorage().setCheckpointVersion(2); Args.getInstance().getStorage().setCheckpointSync(true); @@ -54,9 +57,6 @@ public void removeDb() { Args.clearParam(); context.destroy(); tronDatabase.close(); - FileUtil.deleteDir(new File("output_SnapshotManager_test")); - revokingDatabase.getCheckTmpStore().close(); - tronDatabase.close(); } @Test diff --git a/framework/src/test/java/org/tron/core/db2/RevokingDbWithCacheNewValueTest.java b/framework/src/test/java/org/tron/core/db2/RevokingDbWithCacheNewValueTest.java index 2290df86978..f371f2348a7 100644 --- a/framework/src/test/java/org/tron/core/db2/RevokingDbWithCacheNewValueTest.java +++ b/framework/src/test/java/org/tron/core/db2/RevokingDbWithCacheNewValueTest.java @@ -1,22 +1,22 @@ package org.tron.core.db2; -import java.io.File; +import java.io.IOException; import java.util.Arrays; import java.util.List; import java.util.Set; import java.util.stream.Collectors; import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang3.ArrayUtils; -import org.apache.commons.lang3.RandomStringUtils; import org.junit.After; import org.junit.Assert; import org.junit.Before; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.TemporaryFolder; import org.tron.common.application.Application; import org.tron.common.application.ApplicationFactory; import org.tron.common.application.TronApplicationContext; import org.tron.common.utils.ByteArray; -import org.tron.common.utils.FileUtil; import org.tron.common.utils.SessionOptional; import org.tron.core.Constant; import org.tron.core.capsule.utils.MarketUtils; @@ -34,12 +34,14 @@ public class RevokingDbWithCacheNewValueTest { private TronApplicationContext context; private Application appT; private TestRevokingTronStore tronDatabase; + @Rule + public final TemporaryFolder temporaryFolder = new TemporaryFolder(); private String databasePath = ""; @Before - public void init() { - databasePath = "output_revokingStore_test_" + RandomStringUtils.randomAlphanumeric(10); + public void init() throws IOException { + databasePath = temporaryFolder.newFolder().toString(); Args.setParam(new String[]{"-d", databasePath}, Constant.TEST_CONF); context = new TronApplicationContext(DefaultConfig.class); @@ -51,7 +53,6 @@ public void removeDb() { Args.clearParam(); context.destroy(); tronDatabase.close(); - FileUtil.deleteDir(new File(databasePath)); } @Test diff --git a/framework/src/test/java/org/tron/core/db2/SnapshotImplTest.java b/framework/src/test/java/org/tron/core/db2/SnapshotImplTest.java index aab6f656b1f..649056aa151 100644 --- a/framework/src/test/java/org/tron/core/db2/SnapshotImplTest.java +++ b/framework/src/test/java/org/tron/core/db2/SnapshotImplTest.java @@ -3,16 +3,16 @@ import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNull; -import java.io.File; +import java.io.IOException; import java.lang.reflect.Constructor; import org.junit.After; -import org.junit.Assert; import org.junit.Before; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.TemporaryFolder; import org.tron.common.application.Application; import org.tron.common.application.ApplicationFactory; import org.tron.common.application.TronApplicationContext; -import org.tron.common.utils.FileUtil; import org.tron.core.Constant; import org.tron.core.config.DefaultConfig; import org.tron.core.config.args.Args; @@ -26,10 +26,12 @@ public class SnapshotImplTest { private TronApplicationContext context; private Application appT; private SnapshotManager revokingDatabase; + @Rule + public final TemporaryFolder temporaryFolder = new TemporaryFolder(); @Before - public void init() { - Args.setParam(new String[]{"-d", "output_revokingStore_test"}, Constant.TEST_CONF); + public void init() throws IOException { + Args.setParam(new String[]{"-d", temporaryFolder.newFolder().toString()}, Constant.TEST_CONF); context = new TronApplicationContext(DefaultConfig.class); appT = ApplicationFactory.create(context); @@ -44,10 +46,7 @@ public void init() { public void removeDb() { Args.clearParam(); context.destroy(); - FileUtil.deleteDir(new File("output_revokingStore_test")); - tronDatabase.close(); - revokingDatabase.shutdown(); } /** diff --git a/framework/src/test/java/org/tron/core/db2/SnapshotManagerTest.java b/framework/src/test/java/org/tron/core/db2/SnapshotManagerTest.java index 134dc99e51c..d6fd319e6a0 100644 --- a/framework/src/test/java/org/tron/core/db2/SnapshotManagerTest.java +++ b/framework/src/test/java/org/tron/core/db2/SnapshotManagerTest.java @@ -6,7 +6,7 @@ import com.google.common.collect.Maps; import com.google.common.primitives.Longs; import com.google.protobuf.ByteString; -import java.io.File; +import java.io.IOException; import java.util.Arrays; import java.util.List; import java.util.Map; @@ -15,11 +15,12 @@ import org.junit.After; import org.junit.Assert; import org.junit.Before; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.TemporaryFolder; import org.tron.common.application.Application; import org.tron.common.application.ApplicationFactory; import org.tron.common.application.TronApplicationContext; -import org.tron.common.utils.FileUtil; import org.tron.common.utils.Sha256Hash; import org.tron.core.Constant; import org.tron.core.capsule.BlockCapsule; @@ -40,11 +41,13 @@ public class SnapshotManagerTest { private TronApplicationContext context; private Application appT; private TestRevokingTronStore tronDatabase; + @Rule + public final TemporaryFolder temporaryFolder = new TemporaryFolder(); @Before - public void init() { - Args.setParam(new String[]{"-d", "output_SnapshotManager_test"}, + public void init() throws IOException { + Args.setParam(new String[]{"-d", temporaryFolder.newFolder().toString()}, Constant.TEST_CONF); context = new TronApplicationContext(DefaultConfig.class); appT = ApplicationFactory.create(context); @@ -59,9 +62,6 @@ public void removeDb() { Args.clearParam(); context.destroy(); tronDatabase.close(); - FileUtil.deleteDir(new File("output_SnapshotManager_test")); - revokingDatabase.getCheckTmpStore().close(); - tronDatabase.close(); } @Test diff --git a/framework/src/test/java/org/tron/core/db2/SnapshotRootTest.java b/framework/src/test/java/org/tron/core/db2/SnapshotRootTest.java index 70b4d9eff30..635cc018cc2 100644 --- a/framework/src/test/java/org/tron/core/db2/SnapshotRootTest.java +++ b/framework/src/test/java/org/tron/core/db2/SnapshotRootTest.java @@ -1,10 +1,13 @@ package org.tron.core.db2; import com.google.common.collect.Sets; -import java.io.File; +import java.io.IOException; +import java.nio.file.Path; +import java.nio.file.Paths; import java.util.ArrayList; import java.util.Arrays; import java.util.List; +import java.util.Objects; import java.util.Set; import java.util.stream.Collectors; import lombok.AllArgsConstructor; @@ -13,7 +16,9 @@ import org.junit.After; import org.junit.Assert; import org.junit.Before; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.TemporaryFolder; import org.springframework.util.CollectionUtils; import org.tron.common.application.Application; import org.tron.common.application.ApplicationFactory; @@ -45,11 +50,13 @@ public class SnapshotRootTest { "exchange","market_order","account-trace","contract-state","trans")); private Set allDBNames; private Set allRevokingDBNames; + @Rule + public final TemporaryFolder temporaryFolder = new TemporaryFolder(); @Before - public void init() { - Args.setParam(new String[]{"-d", "output_revokingStore_test"}, Constant.TEST_CONF); + public void init() throws IOException { + Args.setParam(new String[]{"-d", temporaryFolder.newFolder().toString()}, Constant.TEST_CONF); context = new TronApplicationContext(DefaultConfig.class); appT = ApplicationFactory.create(context); } @@ -58,7 +65,6 @@ public void init() { public void removeDb() { Args.clearParam(); context.destroy(); - FileUtil.deleteDir(new File("output_revokingStore_test")); } @Test @@ -133,7 +139,9 @@ public void testSecondCacheCheck() throws ItemNotFoundException { revokingDatabase = context.getBean(SnapshotManager.class); allRevokingDBNames = parseRevokingDBNames(context); - allDBNames = Arrays.stream(new File("output_revokingStore_test/database").list()) + Path path = Paths.get(Args.getInstance().getOutputDirectory(), + Args.getInstance().getStorage().getDbDirectory()); + allDBNames = Arrays.stream(Objects.requireNonNull(path.toFile().list())) .collect(Collectors.toSet()); if (CollectionUtils.isEmpty(allDBNames)) { throw new ItemNotFoundException("No DBs found"); @@ -152,10 +160,13 @@ public void testSecondCacheCheckAddDb() revokingDatabase = context.getBean(SnapshotManager.class); allRevokingDBNames = parseRevokingDBNames(context); allRevokingDBNames.add("secondCheckTestDB"); - FileUtil.createDirIfNotExists("output_revokingStore_test/database/secondCheckTestDB"); - allDBNames = Arrays.stream(new File("output_revokingStore_test/database").list()) + Path path = Paths.get(Args.getInstance().getOutputDirectory(), + Args.getInstance().getStorage().getDbDirectory()); + Path secondCheckTestDB = Paths.get(path.toString(), "secondCheckTestDB"); + FileUtil.createDirIfNotExists(secondCheckTestDB.toString()); + allDBNames = Arrays.stream(Objects.requireNonNull(path.toFile().list())) .collect(Collectors.toSet()); - FileUtil.deleteDir(new File("output_revokingStore_test/database/secondCheckTestDB")); + FileUtil.deleteDir(secondCheckTestDB.toFile()); if (CollectionUtils.isEmpty(allDBNames)) { throw new ItemNotFoundException("No DBs found"); } diff --git a/framework/src/test/java/org/tron/core/event/BlockEventCacheTest.java b/framework/src/test/java/org/tron/core/event/BlockEventCacheTest.java index e99433db3c6..82c887fad53 100644 --- a/framework/src/test/java/org/tron/core/event/BlockEventCacheTest.java +++ b/framework/src/test/java/org/tron/core/event/BlockEventCacheTest.java @@ -13,6 +13,8 @@ public class BlockEventCacheTest { @Test public void test() throws Exception { + BlockCapsule.BlockId solidID = new BlockCapsule.BlockId(getBlockId(), 100); + BlockEvent be1 = new BlockEvent(); BlockCapsule.BlockId b1 = new BlockCapsule.BlockId(getBlockId(), 1); be1.setBlockId(b1); @@ -36,32 +38,46 @@ public void test() throws Exception { BlockEventCache.init(b1); + BlockEvent event = new BlockEvent(); + BlockCapsule.BlockId blockId = new BlockCapsule.BlockId(getBlockId(), 2); + event.setBlockId(blockId); + event.setParentId(b1); + event.setSolidId(blockId); + BlockEventCache.add(event); + Assert.assertEquals(event, BlockEventCache.getHead()); + Assert.assertEquals(blockId, BlockEventCache.getSolidId()); + Assert.assertEquals(event, BlockEventCache.getBlockEvent(blockId)); + + BlockEventCache.init(b1); + BlockEvent be2 = new BlockEvent(); BlockCapsule.BlockId b2 = new BlockCapsule.BlockId(getBlockId(), 2); be2.setBlockId(b2); be2.setParentId(b1); - be2.setSolidId(b1); + be2.setSolidId(solidID); BlockEventCache.add(be2); Assert.assertEquals(be2, BlockEventCache.getHead()); + Assert.assertEquals(b2, BlockEventCache.getSolidId()); Assert.assertEquals(be2, BlockEventCache.getBlockEvent(b2)); BlockEvent be22 = new BlockEvent(); BlockCapsule.BlockId b22 = new BlockCapsule.BlockId(getBlockId(), 2); be22.setBlockId(b22); be22.setParentId(b1); - be22.setSolidId(b22); + be22.setSolidId(solidID); BlockEventCache.add(be22); Assert.assertEquals(be2, BlockEventCache.getHead()); Assert.assertEquals(be22, BlockEventCache.getBlockEvent(b22)); - Assert.assertEquals(b22, BlockEventCache.getSolidId()); + Assert.assertEquals(b2, BlockEventCache.getSolidId()); BlockEvent be3 = new BlockEvent(); BlockCapsule.BlockId b3 = new BlockCapsule.BlockId(getBlockId(), 3); be3.setBlockId(b3); be3.setParentId(b22); - be3.setSolidId(b22); + be3.setSolidId(solidID); BlockEventCache.add(be3); Assert.assertEquals(be3, BlockEventCache.getHead()); + Assert.assertEquals(b3, BlockEventCache.getSolidId()); List list = BlockEventCache.getSolidBlockEvents(b2); Assert.assertEquals(1, list.size()); diff --git a/framework/src/test/java/org/tron/core/event/BlockEventGetTest.java b/framework/src/test/java/org/tron/core/event/BlockEventGetTest.java index b6835cfcf82..9ee46118c14 100644 --- a/framework/src/test/java/org/tron/core/event/BlockEventGetTest.java +++ b/framework/src/test/java/org/tron/core/event/BlockEventGetTest.java @@ -11,13 +11,15 @@ import java.util.List; import java.util.concurrent.atomic.AtomicInteger; import lombok.extern.slf4j.Slf4j; -import org.junit.After; +import org.junit.AfterClass; import org.junit.Assert; import org.junit.Before; +import org.junit.BeforeClass; import org.junit.ClassRule; import org.junit.Test; import org.junit.rules.TemporaryFolder; import org.mockito.Mockito; +import org.tron.api.GrpcAPI; import org.tron.common.application.TronApplicationContext; import org.tron.common.logsfilter.EventPluginConfig; import org.tron.common.logsfilter.EventPluginLoader; @@ -63,26 +65,33 @@ public class BlockEventGetTest extends BlockGenerate { protected Manager dbManager; long currentHeader = -1; private TronNetDelegate tronNetDelegate; - private TronApplicationContext context; + private static TronApplicationContext context; static LocalDateTime localDateTime = LocalDateTime.now(); private long time = ZonedDateTime.of(localDateTime, ZoneId.systemDefault()).toInstant().toEpochMilli(); - protected void initDbPath() throws IOException { - dbPath = temporaryFolder.newFolder().toString(); + + public static String dbPath() { + try { + return temporaryFolder.newFolder().toString(); + } catch (IOException e) { + Assert.fail("create temp folder failed"); + } + return null; + } + + @BeforeClass + public static void init() { + Args.setParam(new String[] {"--output-directory", dbPath()}, Constant.TEST_CONF); + context = new TronApplicationContext(DefaultConfig.class); } @Before public void before() throws IOException { - initDbPath(); - logger.info("Full node running."); - Args.setParam(new String[] {"-d", dbPath}, Constant.TEST_CONF); Args.getInstance().setNodeListenPort(10000 + port.incrementAndGet()); - context = new TronApplicationContext(DefaultConfig.class); - dbManager = context.getBean(Manager.class); setManager(dbManager); @@ -91,7 +100,7 @@ public void before() throws IOException { tronNetDelegate = context.getBean(TronNetDelegate.class); tronNetDelegate.setExit(false); currentHeader = dbManager.getDynamicPropertiesStore() - .getLatestBlockHeaderNumberFromDB(); + .getLatestBlockHeaderNumberFromDB(); ByteString addressBS = ByteString.copyFrom(address); WitnessCapsule witnessCapsule = new WitnessCapsule(addressBS); @@ -108,8 +117,10 @@ public void before() throws IOException { dps.saveAllowTvmShangHai(1); } - @After - public void after() throws IOException { + @AfterClass + public static void after() throws IOException { + context.destroy(); + Args.clearParam(); } @Test @@ -132,13 +143,13 @@ public void test() throws Exception { + "57c973388f044038eff0e6474425b38037e75e66d6b3047647290605449c7764736f6c63430008140033"; Protocol.Transaction trx = TvmTestUtils.generateDeploySmartContractAndGetTransaction( "TestTRC20", address, "[{\"anonymous\":false,\"inputs\":[{\"indexed\":true,\"name\"" - + ":\"from\",\"type\":\"address\"},{\"indexed\":true,\"name\":\"to\",\"type\":\"address\"}" - + ",{\"indexed\":false,\"name\":\"value\",\"type\":\"uint256\"}],\"name\":\"Transfer\"," - + "\"type\":\"event\"}]", code, 0, (long) 1e9, 100, null, 1); + + ":\"from\",\"type\":\"address\"},{\"indexed\":true,\"name\":\"to\",\"type\"" + + ":\"address\"},{\"indexed\":false,\"name\":\"value\",\"type\":\"uint256\"}],\"name\"" + + ":\"Transfer\",\"type\":\"event\"}]", code, 0, (long) 1e9, 100, null, 1); trx = trx.toBuilder().addRet( - Protocol.Transaction.Result.newBuilder() - .setContractRetValue(Protocol.Transaction.Result.contractResult.SUCCESS_VALUE) - .build()).build(); + Protocol.Transaction.Result.newBuilder() + .setContractRetValue(Protocol.Transaction.Result.contractResult.SUCCESS_VALUE) + .build()).build(); Protocol.Block block = getSignedBlock(witnessCapsule.getAddress(), time, privateKey); BlockCapsule blockCapsule = new BlockCapsule(block.toBuilder().addTransactions(trx).build()); @@ -223,6 +234,11 @@ public void getTransactionTriggers() throws Exception { bc = new BlockCapsule(100, ByteString.empty(), 1, transactionList); + Manager manager = mock(Manager.class); + ReflectUtils.setFieldValue(blockEventGet, "manager", manager); + Mockito.when(manager.getTransactionInfoByBlockNum(1)) + .thenReturn(GrpcAPI.TransactionInfoList.newBuilder().build()); + list = blockEventGet.getTransactionTriggers(bc, 1); Assert.assertEquals(1, list.size()); Assert.assertEquals(100, list.get(0).getTransactionLogTrigger().getTimeStamp()); @@ -243,21 +259,14 @@ public void getTransactionTriggers() throws Exception { .addContractResult(ByteString.copyFrom(ByteArray.fromHexString("112233"))) .setReceipt(resourceBuild.build()); - Manager manager = mock(Manager.class); - ReflectUtils.setFieldValue(blockEventGet, "manager", manager); - ChainBaseManager chainBaseManager = mock(ChainBaseManager.class); Mockito.when(manager.getChainBaseManager()).thenReturn(chainBaseManager); - TransactionRetStore transactionRetStore = mock(TransactionRetStore.class); - Mockito.when(chainBaseManager.getTransactionRetStore()).thenReturn(transactionRetStore); - - Protocol.TransactionRet transactionRet = Protocol.TransactionRet.newBuilder() - .addTransactioninfo(infoBuild.build()).build(); - TransactionRetCapsule result = new TransactionRetCapsule(transactionRet.toByteArray()); + GrpcAPI.TransactionInfoList result = GrpcAPI.TransactionInfoList.newBuilder() + .addTransactionInfo(infoBuild.build()).build(); - Mockito.when(transactionRetStore.getTransactionInfoByBlockNum(ByteArray.fromLong(0))) + Mockito.when(manager.getTransactionInfoByBlockNum(0)) .thenReturn(result); Protocol.Block block = Protocol.Block.newBuilder() @@ -276,8 +285,8 @@ public void getTransactionTriggers() throws Exception { Assert.assertEquals(2, list.get(0).getTransactionLogTrigger().getEnergyUsageTotal()); - Mockito.when(transactionRetStore.getTransactionInfoByBlockNum(ByteArray.fromLong(0))) - .thenReturn(null); + Mockito.when(manager.getTransactionInfoByBlockNum(0)) + .thenReturn(GrpcAPI.TransactionInfoList.newBuilder().build()); list = blockEventGet.getTransactionTriggers(blockCapsule, 1); Assert.assertEquals(1, list.size()); Assert.assertEquals(1, diff --git a/framework/src/test/java/org/tron/core/event/HistoryEventServiceTest.java b/framework/src/test/java/org/tron/core/event/HistoryEventServiceTest.java index 49f77ccf597..1485d726235 100644 --- a/framework/src/test/java/org/tron/core/event/HistoryEventServiceTest.java +++ b/framework/src/test/java/org/tron/core/event/HistoryEventServiceTest.java @@ -23,7 +23,7 @@ public class HistoryEventServiceTest { HistoryEventService historyEventService = new HistoryEventService(); - @Test + @Test(timeout = 60_000) public void test() throws Exception { EventPluginLoader instance = mock(EventPluginLoader.class); Mockito.when(instance.isUseNativeQueue()).thenReturn(true); @@ -43,6 +43,7 @@ public void test() throws Exception { RealtimeEventService realtimeEventService = new RealtimeEventService(); BlockEventLoad blockEventLoad = new BlockEventLoad(); ReflectUtils.setFieldValue(blockEventLoad, "instance", instance); + ReflectUtils.setFieldValue(blockEventLoad, "manager", manager); ReflectUtils.setFieldValue(historyEventService, "solidEventService", solidEventService); ReflectUtils.setFieldValue(historyEventService, "realtimeEventService", realtimeEventService); diff --git a/framework/src/test/java/org/tron/core/exception/TronErrorTest.java b/framework/src/test/java/org/tron/core/exception/TronErrorTest.java index b4c3dc4b07f..39fe1404b18 100644 --- a/framework/src/test/java/org/tron/core/exception/TronErrorTest.java +++ b/framework/src/test/java/org/tron/core/exception/TronErrorTest.java @@ -2,17 +2,25 @@ import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertThrows; +import static org.junit.Assert.fail; import static org.mockito.ArgumentMatchers.any; import static org.mockito.Mockito.mockStatic; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.times; +import ch.qos.logback.classic.LoggerContext; +import ch.qos.logback.classic.util.ContextInitializer; +import ch.qos.logback.core.joran.spi.JoranException; import com.typesafe.config.Config; import com.typesafe.config.ConfigFactory; import com.typesafe.config.ConfigObject; import java.io.IOException; +import java.lang.reflect.Field; import java.nio.file.Path; import java.util.Collections; import java.util.HashMap; import java.util.Map; +import java.util.concurrent.atomic.AtomicBoolean; import org.junit.After; import org.junit.Assert; import org.junit.Rule; @@ -22,6 +30,8 @@ import org.mockito.MockedStatic; import org.mockito.Mockito; import org.mockito.junit.MockitoJUnitRunner; +import org.slf4j.LoggerFactory; +import org.tron.common.arch.Arch; import org.tron.common.log.LogService; import org.tron.common.parameter.RateLimiterInitialization; import org.tron.common.utils.ReflectUtils; @@ -32,6 +42,7 @@ import org.tron.core.services.http.RateLimiterServlet; import org.tron.core.zen.ZksnarkInitService; + @RunWith(MockitoJUnitRunner.class) public class TronErrorTest { @@ -39,7 +50,7 @@ public class TronErrorTest { public TemporaryFolder temporaryFolder = new TemporaryFolder(); @After - public void clearMocks() { + public void clearMocks() { Mockito.clearAllCaches(); Args.clearParam(); } @@ -57,9 +68,14 @@ public void testTronError() { } @Test - public void ZksnarkInitTest() { + public void ZksnarkInitTest() throws IllegalAccessException, NoSuchFieldException { + Field field = ZksnarkInitService.class.getDeclaredField("initialized"); + field.setAccessible(true); + AtomicBoolean atomicBoolean = (AtomicBoolean) field.get(null); + boolean originalValue = atomicBoolean.get(); + atomicBoolean.set(false); + try (MockedStatic mock = mockStatic(JLibrustzcash.class)) { - mock.when(JLibrustzcash::isOpenZen).thenReturn(true); mock.when(() -> JLibrustzcash.librustzcashInitZksnarkParams(any())) .thenAnswer(invocation -> { throw new ZksnarkException("Zksnark init failed"); @@ -67,15 +83,29 @@ public void ZksnarkInitTest() { TronError thrown = assertThrows(TronError.class, ZksnarkInitService::librustzcashInitZksnarkParams); assertEquals(TronError.ErrCode.ZCASH_INIT, thrown.getErrCode()); + } finally { + atomicBoolean.set(originalValue); } } @Test public void LogLoadTest() throws IOException { - LogService.load("non-existent.xml"); - Path path = temporaryFolder.newFile("logback.xml").toPath(); - TronError thrown = assertThrows(TronError.class, () -> LogService.load(path.toString())); - assertEquals(TronError.ErrCode.LOG_LOAD, thrown.getErrCode()); + LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory(); + + try { + LogService.load("non-existent.xml"); + Path path = temporaryFolder.newFile("logback.xml").toPath(); + TronError thrown = assertThrows(TronError.class, () -> LogService.load(path.toString())); + assertEquals(TronError.ErrCode.LOG_LOAD, thrown.getErrCode()); + } finally { + try { + context.reset(); + ContextInitializer ci = new ContextInitializer(context); + ci.autoConfig(); + } catch (JoranException e) { + Assert.fail(e.getMessage()); + } + } } @Test @@ -106,7 +136,7 @@ public void rateLimiterServletInitTest() { @Test public void shutdownBlockTimeInitTest() { - Map params = new HashMap<>(); + Map params = new HashMap<>(); params.put(Constant.NODE_SHUTDOWN_BLOCK_TIME, "0"); params.put("storage.db.directory", "database"); Config config = ConfigFactory.defaultOverrides().withFallback( @@ -114,4 +144,58 @@ public void shutdownBlockTimeInitTest() { TronError thrown = assertThrows(TronError.class, () -> Args.setParam(config)); assertEquals(TronError.ErrCode.AUTO_STOP_PARAMS, thrown.getErrCode()); } + + @Test + public void testThrowIfUnsupportedJavaVersion() { + runArchTest("x86_64", "1.8", false); + runArchTest("x86_64", "11", true); + runArchTest("x86_64", "17", true); + runArchTest("aarch64", "17", false); + runArchTest("aarch64", "1.8", true); + runArchTest("aarch64", "11", true); + } + + private void runArchTest(String osArch, String javaVersion, boolean expectThrow) { + try (MockedStatic mocked = mockStatic(Arch.class)) { + boolean isX86 = "x86_64".equals(osArch); + boolean isArm64 = "aarch64".equals(osArch); + + boolean isJava8 = "1.8".equals(javaVersion); + boolean isJava17 = "17".equals(javaVersion); + + mocked.when(Arch::isX86).thenReturn(isX86); + mocked.when(Arch::isArm64).thenReturn(isArm64); + + mocked.when(Arch::isJava8).thenReturn(isJava8); + mocked.when(Arch::isJava17).thenReturn(isJava17); + + mocked.when(Arch::getOsArch).thenReturn(osArch); + mocked.when(Arch::javaSpecificationVersion).thenReturn(javaVersion); + mocked.when(Arch::withAll).thenReturn(String.format( + "Architecture: %s, Java Version: %s", osArch, javaVersion)); + + mocked.when(Arch::throwIfUnsupportedJavaVersion).thenCallRealMethod(); + + if (expectThrow) { + TronError err = assertThrows( + TronError.class, () -> Args.setParam(new String[]{}, Constant.TEST_CONF)); + + String expectedJavaVersion = isX86 ? "1.8" : "17"; + String expectedMessage = String.format( + "Java %s is required for %s architecture. Detected version %s", + expectedJavaVersion, osArch, javaVersion); + assertEquals(expectedMessage, err.getCause().getMessage()); + assertEquals(TronError.ErrCode.JDK_VERSION, err.getErrCode()); + mocked.verify(Arch::withAll, times(1)); + } else { + try { + Arch.throwIfUnsupportedJavaVersion(); + } catch (Exception e) { + fail("Expected no exception, but got: " + e.getMessage()); + } + mocked.verify(Arch::withAll, never()); + } + } + } + } diff --git a/framework/src/test/java/org/tron/core/jsonrpc/ApiUtilTest.java b/framework/src/test/java/org/tron/core/jsonrpc/ApiUtilTest.java index 570e7ed3498..f62d47d5367 100644 --- a/framework/src/test/java/org/tron/core/jsonrpc/ApiUtilTest.java +++ b/framework/src/test/java/org/tron/core/jsonrpc/ApiUtilTest.java @@ -4,10 +4,13 @@ import static org.tron.keystore.Wallet.generateRandomBytes; import com.google.protobuf.ByteString; +import org.junit.AfterClass; import org.junit.Assert; +import org.junit.BeforeClass; import org.junit.Test; import org.tron.common.utils.ByteArray; import org.tron.core.capsule.BlockCapsule; +import org.tron.core.config.args.Args; import org.tron.core.services.jsonrpc.JsonRpcApiUtil; import org.tron.protos.Protocol.Block; import org.tron.protos.Protocol.BlockHeader; @@ -16,6 +19,16 @@ public class ApiUtilTest { + @BeforeClass + public static void init() { + Args.setParam(new String[]{}, "config-localtest.conf"); + } + + @AfterClass + public static void clear() { + Args.clearParam(); + } + @Test public void testGetBlockID() { byte[] mockedHash = generateRandomBytes(128); diff --git a/framework/src/test/java/org/tron/core/jsonrpc/BloomTest.java b/framework/src/test/java/org/tron/core/jsonrpc/BloomTest.java index 19e232b0f93..c88615d2d2f 100644 --- a/framework/src/test/java/org/tron/core/jsonrpc/BloomTest.java +++ b/framework/src/test/java/org/tron/core/jsonrpc/BloomTest.java @@ -131,7 +131,7 @@ private TransactionInfo createTransactionInfo(byte[] address1, byte[] address2) @Test public void benchmarkCreateByTransaction() { - int times = 10000; + int times = 1000; // small TransactionRetCapsule smallCapsule = new TransactionRetCapsule(); diff --git a/framework/src/test/java/org/tron/core/jsonrpc/ConcurrentHashMapTest.java b/framework/src/test/java/org/tron/core/jsonrpc/ConcurrentHashMapTest.java index 1de106f68e8..abb73671161 100644 --- a/framework/src/test/java/org/tron/core/jsonrpc/ConcurrentHashMapTest.java +++ b/framework/src/test/java/org/tron/core/jsonrpc/ConcurrentHashMapTest.java @@ -31,7 +31,7 @@ private static int randomInt(int minInt, int maxInt) { */ @Test public void testHandleBlockHash() { - int times = 200; + int times = 100; int eachCount = 200; Map conMap = TronJsonRpcImpl.getBlockFilter2ResultFull(); @@ -67,7 +67,7 @@ public void run() { TronJsonRpcImpl.handleBLockFilter(blockFilterCapsule); } try { - Thread.sleep(randomInt(100, 200)); + Thread.sleep(randomInt(50, 100)); } catch (InterruptedException e) { e.printStackTrace(); } @@ -81,7 +81,7 @@ public void run() { for (int t = 1; t <= times * 2; t++) { try { - Thread.sleep(randomInt(100, 200)); + Thread.sleep(randomInt(50, 100)); } catch (InterruptedException e) { e.printStackTrace(); } @@ -111,7 +111,7 @@ public void run() { for (int t = 1; t <= times * 2; t++) { try { - Thread.sleep(randomInt(100, 200)); + Thread.sleep(randomInt(50, 100)); } catch (InterruptedException e) { e.printStackTrace(); } @@ -144,7 +144,7 @@ public void run() { for (int t = 1; t <= times * 2; t++) { try { - Thread.sleep(randomInt(100, 200)); + Thread.sleep(randomInt(50, 100)); } catch (InterruptedException e) { e.printStackTrace(); } diff --git a/framework/src/test/java/org/tron/core/jsonrpc/JsonRpcTest.java b/framework/src/test/java/org/tron/core/jsonrpc/JsonRpcTest.java index bef0b5a1593..bd357101da3 100644 --- a/framework/src/test/java/org/tron/core/jsonrpc/JsonRpcTest.java +++ b/framework/src/test/java/org/tron/core/jsonrpc/JsonRpcTest.java @@ -18,7 +18,7 @@ import org.tron.common.utils.ByteArray; import org.tron.common.utils.ByteUtil; import org.tron.common.utils.Commons; -import org.tron.core.exception.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; import org.tron.core.services.jsonrpc.JsonRpcApiUtil; import org.tron.core.services.jsonrpc.TronJsonRpc.FilterRequest; import org.tron.core.services.jsonrpc.filters.LogBlockQuery; diff --git a/framework/src/test/java/org/tron/core/jsonrpc/JsonrpcServiceTest.java b/framework/src/test/java/org/tron/core/jsonrpc/JsonrpcServiceTest.java index 0f2214c5c9c..81db38ce286 100644 --- a/framework/src/test/java/org/tron/core/jsonrpc/JsonrpcServiceTest.java +++ b/framework/src/test/java/org/tron/core/jsonrpc/JsonrpcServiceTest.java @@ -1,6 +1,7 @@ package org.tron.core.jsonrpc; import static org.tron.core.services.jsonrpc.JsonRpcApiUtil.getByJsonBlockId; +import static org.tron.core.services.jsonrpc.TronJsonRpcImpl.TAG_PENDING_SUPPORT_ERROR; import com.alibaba.fastjson.JSON; import com.google.gson.JsonArray; @@ -33,9 +34,12 @@ import org.tron.core.capsule.AccountCapsule; import org.tron.core.capsule.BlockCapsule; import org.tron.core.capsule.TransactionCapsule; +import org.tron.core.capsule.TransactionInfoCapsule; +import org.tron.core.capsule.TransactionRetCapsule; import org.tron.core.capsule.utils.BlockUtil; import org.tron.core.config.args.Args; -import org.tron.core.exception.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInternalException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; import org.tron.core.services.NodeInfoService; import org.tron.core.services.interfaceJsonRpcOnPBFT.JsonRpcServiceOnPBFT; import org.tron.core.services.interfaceJsonRpcOnSolidity.JsonRpcServiceOnSolidity; @@ -44,6 +48,7 @@ import org.tron.core.services.jsonrpc.TronJsonRpcImpl; import org.tron.core.services.jsonrpc.filters.LogFilterWrapper; import org.tron.core.services.jsonrpc.types.BlockResult; +import org.tron.core.services.jsonrpc.types.TransactionReceipt; import org.tron.core.services.jsonrpc.types.TransactionResult; import org.tron.protos.Protocol; import org.tron.protos.Protocol.Transaction.Contract.ContractType; @@ -127,6 +132,7 @@ public void init() { (Wallet.getAddressPreFixString() + "ED738B3A0FE390EAA71B768B6D02CDBD18FB207B")))) .build(); + transactionCapsule1 = new TransactionCapsule(transferContract1, ContractType.TransferContract); transactionCapsule1.setBlockNum(blockCapsule1.getNum()); TransactionCapsule transactionCapsule2 = new TransactionCapsule(transferContract2, @@ -155,6 +161,42 @@ public void init() { dbManager.getTransactionStore() .put(transactionCapsule3.getTransactionId().getBytes(), transactionCapsule3); + dbManager.getTransactionStore() + .put(transactionCapsule3.getTransactionId().getBytes(), transactionCapsule3); + + List logs = new ArrayList<>(); + logs.add(Protocol.TransactionInfo.Log.newBuilder() + .setAddress(ByteString.copyFrom("address1".getBytes())) + .setData(ByteString.copyFrom("data1".getBytes())) + .addTopics(ByteString.copyFrom("topic1".getBytes())) + .build()); + logs.add(Protocol.TransactionInfo.Log.newBuilder() + .setAddress(ByteString.copyFrom("address2".getBytes())) + .setData(ByteString.copyFrom("data2".getBytes())) + .addTopics(ByteString.copyFrom("topic2".getBytes())) + .build()); + + TransactionRetCapsule transactionRetCapsule1 = new TransactionRetCapsule(); + blockCapsule1.getTransactions().forEach(tx -> { + TransactionInfoCapsule transactionInfoCapsule = new TransactionInfoCapsule(); + transactionInfoCapsule.setId(tx.getTransactionId().getBytes()); + transactionInfoCapsule.setBlockNumber(blockCapsule1.getNum()); + transactionInfoCapsule.addAllLog(logs); + transactionRetCapsule1.addTransactionInfo(transactionInfoCapsule.getInstance()); + }); + dbManager.getTransactionRetStore() + .put(ByteArray.fromLong(blockCapsule1.getNum()), transactionRetCapsule1); + + TransactionRetCapsule transactionRetCapsule2 = new TransactionRetCapsule(); + blockCapsule2.getTransactions().forEach(tx -> { + TransactionInfoCapsule transactionInfoCapsule = new TransactionInfoCapsule(); + transactionInfoCapsule.setId(tx.getTransactionId().getBytes()); + transactionInfoCapsule.setBlockNumber(blockCapsule2.getNum()); + transactionRetCapsule2.addTransactionInfo(transactionInfoCapsule.getInstance()); + }); + dbManager.getTransactionRetStore() + .put(ByteArray.fromLong(blockCapsule2.getNum()), transactionRetCapsule2); + tronJsonRpc = new TronJsonRpcImpl(nodeInfoService, wallet, dbManager); } @@ -281,6 +323,8 @@ public void testGetBlockByNumber() { } Assert.assertEquals(ByteArray.toJsonHex(0L), blockResult.getNumber()); Assert.assertEquals(ByteArray.toJsonHex(blockCapsule0.getNum()), blockResult.getNumber()); + Assert.assertEquals(blockResult.getTransactions().length, + blockCapsule0.getTransactions().size()); // latest try { @@ -434,7 +478,8 @@ public void testGetByJsonBlockId() { getByJsonBlockId("0xxabc", wallet); Assert.fail("Expected to be thrown"); } catch (Exception e) { - Assert.assertEquals("For input string: \"xabc\"", e.getMessage()); + // https://bugs.openjdk.org/browse/JDK-8176425, from JDK 12, the exception message is changed + Assert.assertTrue(e.getMessage().startsWith("For input string: \"xabc\"")); } } @@ -968,4 +1013,90 @@ public void testNewFilterFinalizedBlock() { Assert.assertEquals("invalid block range params", e.getMessage()); } } + + @Test + public void testGetBlockReceipts() { + + try { + List transactionReceiptList = tronJsonRpc.getBlockReceipts("0x2710"); + Assert.assertFalse(transactionReceiptList.isEmpty()); + for (TransactionReceipt transactionReceipt: transactionReceiptList) { + TransactionReceipt transactionReceipt1 + = tronJsonRpc.getTransactionReceipt(transactionReceipt.getTransactionHash()); + + Assert.assertEquals( + JSON.toJSONString(transactionReceipt), JSON.toJSONString(transactionReceipt1)); + } + } catch (JsonRpcInvalidParamsException | JsonRpcInternalException e) { + throw new RuntimeException(e); + } + + try { + List transactionReceiptList = tronJsonRpc.getBlockReceipts("earliest"); + Assert.assertNull(transactionReceiptList); + } catch (JsonRpcInvalidParamsException | JsonRpcInternalException e) { + throw new RuntimeException(e); + } + + try { + List transactionReceiptList = tronJsonRpc.getBlockReceipts("latest"); + Assert.assertFalse(transactionReceiptList.isEmpty()); + } catch (JsonRpcInvalidParamsException | JsonRpcInternalException e) { + throw new RuntimeException(e); + } + + try { + List transactionReceiptList = tronJsonRpc.getBlockReceipts("finalized"); + Assert.assertFalse(transactionReceiptList.isEmpty()); + } catch (JsonRpcInvalidParamsException | JsonRpcInternalException e) { + throw new RuntimeException(e); + } + + try { + tronJsonRpc.getBlockReceipts("pending"); + Assert.fail(); + } catch (Exception e) { + Assert.assertEquals(TAG_PENDING_SUPPORT_ERROR, e.getMessage()); + } + + try { + tronJsonRpc.getBlockReceipts("test"); + Assert.fail(); + } catch (Exception e) { + Assert.assertEquals("invalid block number", e.getMessage()); + } + + try { + List transactionReceiptList = tronJsonRpc.getBlockReceipts("0x2"); + Assert.assertNull(transactionReceiptList); + } catch (JsonRpcInvalidParamsException | JsonRpcInternalException e) { + throw new RuntimeException(e); + } + + try { + String blockHash = blockCapsule1.getBlockId().toString(); + List transactionReceiptList + = tronJsonRpc.getBlockReceipts(blockHash); + List transactionReceiptList2 + = tronJsonRpc.getBlockReceipts("0x" + blockHash); + + Assert.assertFalse(transactionReceiptList.isEmpty()); + Assert.assertEquals(JSON.toJSONString(transactionReceiptList), + JSON.toJSONString(transactionReceiptList2)); + } catch (JsonRpcInvalidParamsException | JsonRpcInternalException e) { + throw new RuntimeException(e); + } + + } + + @Test + public void testWeb3ClientVersion() { + try { + String[] versions = tronJsonRpc.web3ClientVersion().split("/"); + String javaVersion = versions[versions.length - 1]; + Assert.assertTrue("Java1.8".equals(javaVersion) || "Java17".equals(javaVersion)); + } catch (Exception e) { + Assert.fail(); + } + } } diff --git a/framework/src/test/java/org/tron/core/jsonrpc/LogBlockQueryTest.java b/framework/src/test/java/org/tron/core/jsonrpc/LogBlockQueryTest.java new file mode 100644 index 00000000000..7442e92c8f5 --- /dev/null +++ b/framework/src/test/java/org/tron/core/jsonrpc/LogBlockQueryTest.java @@ -0,0 +1,108 @@ +package org.tron.core.jsonrpc; + +import java.lang.reflect.Method; +import java.util.BitSet; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import javax.annotation.Resource; +import org.junit.After; +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.tron.common.BaseTest; +import org.tron.core.Constant; +import org.tron.core.config.args.Args; +import org.tron.core.services.jsonrpc.TronJsonRpc.FilterRequest; +import org.tron.core.services.jsonrpc.filters.LogBlockQuery; +import org.tron.core.services.jsonrpc.filters.LogFilterWrapper; +import org.tron.core.store.SectionBloomStore; + +public class LogBlockQueryTest extends BaseTest { + + @Resource + SectionBloomStore sectionBloomStore; + private ExecutorService sectionExecutor; + private Method partialMatchMethod; + private static final long CURRENT_MAX_BLOCK_NUM = 50000L; + + static { + Args.setParam(new String[] {"--output-directory", dbPath()}, Constant.TEST_CONF); + } + + @Before + public void setup() throws Exception { + sectionExecutor = Executors.newFixedThreadPool(5); + + // Get private method through reflection + partialMatchMethod = LogBlockQuery.class.getDeclaredMethod("partialMatch", + int[][].class, int.class); + partialMatchMethod.setAccessible(true); + + BitSet bitSet = new BitSet(SectionBloomStore.BLOCK_PER_SECTION); + bitSet.set(0, SectionBloomStore.BLOCK_PER_SECTION); + sectionBloomStore.put(0, 1, bitSet); + sectionBloomStore.put(0, 2, bitSet); + sectionBloomStore.put(0, 3, bitSet); + BitSet bitSet2 = new BitSet(SectionBloomStore.BLOCK_PER_SECTION); + bitSet2.set(0); + sectionBloomStore.put(1, 1, bitSet2); + sectionBloomStore.put(1, 2, bitSet2); + sectionBloomStore.put(1, 3, bitSet2); + } + + @After + public void tearDown() { + if (sectionExecutor != null && !sectionExecutor.isShutdown()) { + sectionExecutor.shutdown(); + } + } + + @Test + public void testPartialMatch() throws Exception { + // Create a basic LogFilterWrapper + LogFilterWrapper logFilterWrapper = new LogFilterWrapper( + new FilterRequest("0x0", "0x1", null, null, null), + CURRENT_MAX_BLOCK_NUM, null, false); + + LogBlockQuery logBlockQuery = new LogBlockQuery(logFilterWrapper, sectionBloomStore, + CURRENT_MAX_BLOCK_NUM, sectionExecutor); + + int section = 0; + + // Create a hit condition + int[][] bitIndexes = new int[][] { + {1, 2, 3}, // topic0 + {4, 5, 6} // topic1 + }; + + // topic0 hit section 0 + BitSet result = (BitSet) partialMatchMethod.invoke(logBlockQuery, bitIndexes, section); + Assert.assertNotNull(result); + Assert.assertEquals(SectionBloomStore.BLOCK_PER_SECTION, result.cardinality()); + + // topic0 hit section 1 + result = (BitSet) partialMatchMethod.invoke(logBlockQuery, bitIndexes, 1); + Assert.assertNotNull(result); + Assert.assertEquals(1, result.cardinality()); + + // not exist section 2 + result = (BitSet) partialMatchMethod.invoke(logBlockQuery, bitIndexes, 2); + Assert.assertNotNull(result); + Assert.assertTrue(result.isEmpty()); + + //not hit + bitIndexes = new int[][] { + {1, 2, 4}, // topic0 + {3, 5, 6} // topic1 + }; + result = (BitSet) partialMatchMethod.invoke(logBlockQuery, bitIndexes, section); + Assert.assertNotNull(result); + Assert.assertTrue(result.isEmpty()); + + // null condition + bitIndexes = new int[0][]; + result = (BitSet) partialMatchMethod.invoke(logBlockQuery, bitIndexes, section); + Assert.assertNotNull(result); + Assert.assertTrue(result.isEmpty()); + } +} \ No newline at end of file diff --git a/framework/src/test/java/org/tron/core/jsonrpc/LogMatchExactlyTest.java b/framework/src/test/java/org/tron/core/jsonrpc/LogMatchExactlyTest.java index 600cc52b58e..0f9f125b74e 100644 --- a/framework/src/test/java/org/tron/core/jsonrpc/LogMatchExactlyTest.java +++ b/framework/src/test/java/org/tron/core/jsonrpc/LogMatchExactlyTest.java @@ -4,12 +4,13 @@ import java.util.ArrayList; import java.util.List; +import java.util.Objects; import org.junit.Assert; import org.junit.Test; import org.tron.common.runtime.vm.DataWord; import org.tron.common.runtime.vm.LogInfo; import org.tron.common.utils.ByteArray; -import org.tron.core.exception.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; import org.tron.core.services.jsonrpc.TronJsonRpc.FilterRequest; import org.tron.core.services.jsonrpc.TronJsonRpc.LogFilterElement; import org.tron.core.services.jsonrpc.filters.LogFilter; @@ -220,6 +221,18 @@ public void testMatchBlock() { List elementList = matchBlock(logFilter, 100, null, transactionInfoList, false); Assert.assertEquals(1, elementList.size()); + + //test LogFilterElement + List elementList2 = + matchBlock(logFilter, 100, null, transactionInfoList, false); + Assert.assertEquals(1, elementList2.size()); + + LogFilterElement logFilterElement1 = elementList.get(0); + LogFilterElement logFilterElement2 = elementList2.get(0); + + Assert.assertEquals(logFilterElement1.hashCode(), logFilterElement2.hashCode()); + Assert.assertEquals(logFilterElement1, logFilterElement2); + } catch (JsonRpcInvalidParamsException e) { Assert.fail(); } diff --git a/framework/src/test/java/org/tron/core/jsonrpc/SectionBloomStoreTest.java b/framework/src/test/java/org/tron/core/jsonrpc/SectionBloomStoreTest.java index 111370bfffd..953465b299e 100644 --- a/framework/src/test/java/org/tron/core/jsonrpc/SectionBloomStoreTest.java +++ b/framework/src/test/java/org/tron/core/jsonrpc/SectionBloomStoreTest.java @@ -235,5 +235,7 @@ public void testWriteAndQuery() { } catch (Exception e) { Assert.fail(); } + + sectionExecutor.shutdownNow(); } } diff --git a/framework/src/test/java/org/tron/core/metrics/prometheus/PrometheusApiServiceTest.java b/framework/src/test/java/org/tron/core/metrics/prometheus/PrometheusApiServiceTest.java index 195e4749209..37c376ce9af 100644 --- a/framework/src/test/java/org/tron/core/metrics/prometheus/PrometheusApiServiceTest.java +++ b/framework/src/test/java/org/tron/core/metrics/prometheus/PrometheusApiServiceTest.java @@ -41,7 +41,7 @@ public class PrometheusApiServiceTest extends BaseTest { static LocalDateTime localDateTime = LocalDateTime.now(); @Resource private DposSlot dposSlot; - final int blocks = 512; + final int blocks = 100; private final String key = PublicMethod.getRandomPrivateKey(); private final byte[] privateKey = ByteArray.fromHexString(key); private static final AtomicInteger port = new AtomicInteger(0); diff --git a/framework/src/test/java/org/tron/core/net/BaseNet.java b/framework/src/test/java/org/tron/core/net/BaseNet.java deleted file mode 100644 index 65771bae952..00000000000 --- a/framework/src/test/java/org/tron/core/net/BaseNet.java +++ /dev/null @@ -1,134 +0,0 @@ -package org.tron.core.net; - -import io.netty.bootstrap.Bootstrap; -import io.netty.channel.Channel; -import io.netty.channel.ChannelInitializer; -import io.netty.channel.ChannelOption; -import io.netty.channel.DefaultMessageSizeEstimator; -import io.netty.channel.FixedRecvByteBufAllocator; -import io.netty.channel.nio.NioEventLoopGroup; -import io.netty.channel.socket.nio.NioSocketChannel; -import io.netty.handler.codec.ByteToMessageDecoder; -import io.netty.handler.codec.protobuf.ProtobufVarint32FrameDecoder; -import io.netty.handler.codec.protobuf.ProtobufVarint32LengthFieldPrepender; -import io.netty.handler.timeout.ReadTimeoutHandler; -import io.netty.handler.timeout.WriteTimeoutHandler; -import java.io.File; -import java.io.IOException; -import java.util.Collection; -import java.util.concurrent.ExecutorService; -import java.util.concurrent.Executors; -import java.util.concurrent.TimeUnit; -import lombok.extern.slf4j.Slf4j; -import org.junit.AfterClass; -import org.junit.Assert; -import org.junit.BeforeClass; -import org.junit.ClassRule; -import org.junit.rules.TemporaryFolder; -import org.tron.common.application.Application; -import org.tron.common.application.ApplicationFactory; -import org.tron.common.application.TronApplicationContext; -import org.tron.common.parameter.CommonParameter; -import org.tron.common.utils.FileUtil; -import org.tron.common.utils.PublicMethod; -import org.tron.common.utils.ReflectUtils; -import org.tron.core.Constant; -import org.tron.core.config.DefaultConfig; -import org.tron.core.config.args.Args; -import org.tron.core.net.peer.PeerConnection; - -@Slf4j -public class BaseNet { - - @ClassRule - public static final TemporaryFolder temporaryFolder = new TemporaryFolder(); - private static String dbDirectory = "net-database"; - private static String indexDirectory = "net-index"; - private static int port = 10000; - - protected static TronApplicationContext context; - - private static Application appT; - private static TronNetDelegate tronNetDelegate; - - private static ExecutorService executorService = Executors.newFixedThreadPool(1); - - public static Channel connect(ByteToMessageDecoder decoder) throws InterruptedException { - NioEventLoopGroup group = new NioEventLoopGroup(1); - Bootstrap b = new Bootstrap(); - b.group(group).channel(NioSocketChannel.class) - .handler(new ChannelInitializer() { - @Override - protected void initChannel(Channel ch) throws Exception { - ch.config().setRecvByteBufAllocator(new FixedRecvByteBufAllocator(256 * 1024)); - ch.config().setOption(ChannelOption.SO_RCVBUF, 256 * 1024); - ch.config().setOption(ChannelOption.SO_BACKLOG, 1024); - ch.pipeline() - .addLast("readTimeoutHandler", new ReadTimeoutHandler(600, TimeUnit.SECONDS)) - .addLast("writeTimeoutHandler", new WriteTimeoutHandler(600, TimeUnit.SECONDS)); - ch.pipeline().addLast("protoPender", new ProtobufVarint32LengthFieldPrepender()); - ch.pipeline().addLast("lengthDecode", new ProtobufVarint32FrameDecoder()); - ch.pipeline().addLast("handshakeHandler", decoder); - ch.closeFuture(); - } - }).option(ChannelOption.SO_KEEPALIVE, true) - .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 60000) - .option(ChannelOption.MESSAGE_SIZE_ESTIMATOR, DefaultMessageSizeEstimator.DEFAULT); - return b.connect(Constant.LOCAL_HOST, port).sync().channel(); - } - - @BeforeClass - public static void init() throws Exception { - executorService.execute(() -> { - logger.info("Full node running."); - try { - Args.setParam( - new String[]{ - "--output-directory", temporaryFolder.newFolder().toString(), - "--storage-db-directory", dbDirectory, - "--storage-index-directory", indexDirectory - }, - "config.conf" - ); - } catch (IOException e) { - Assert.fail("create temp db directory failed"); - } - CommonParameter parameter = Args.getInstance(); - parameter.setNodeListenPort(port); - parameter.getSeedNode().getAddressList().clear(); - parameter.setNodeExternalIp(Constant.LOCAL_HOST); - parameter.setRpcEnable(true); - parameter.setRpcPort(PublicMethod.chooseRandomPort()); - parameter.setRpcSolidityEnable(false); - parameter.setRpcPBFTEnable(false); - parameter.setFullNodeHttpEnable(false); - parameter.setSolidityNodeHttpEnable(false); - parameter.setPBFTHttpEnable(false); - context = new TronApplicationContext(DefaultConfig.class); - appT = ApplicationFactory.create(context); - appT.startup(); - try { - Thread.sleep(2000); - } catch (InterruptedException e) { - //ignore - } - tronNetDelegate = context.getBean(TronNetDelegate.class); - appT.blockUntilShutdown(); - }); - int tryTimes = 0; - do { - Thread.sleep(3000); //coverage consumerInvToSpread,consumerInvToFetch in AdvService.init - } while (++tryTimes < 100 && tronNetDelegate == null); - } - - @AfterClass - public static void destroy() { - Collection peerConnections = ReflectUtils - .invokeMethod(tronNetDelegate, "getActivePeer"); - for (PeerConnection peer : peerConnections) { - peer.getChannel().close(); - } - Args.clearParam(); - context.destroy(); - } -} diff --git a/framework/src/test/java/org/tron/core/net/BaseNetTest.java b/framework/src/test/java/org/tron/core/net/BaseNetTest.java deleted file mode 100644 index fd0847c1f08..00000000000 --- a/framework/src/test/java/org/tron/core/net/BaseNetTest.java +++ /dev/null @@ -1,16 +0,0 @@ -package org.tron.core.net; - -import lombok.extern.slf4j.Slf4j; -import org.junit.Test; -import org.tron.core.services.DelegationServiceTest; -import org.tron.core.services.NodeInfoServiceTest; - -@Slf4j -public class BaseNetTest extends BaseNet { - - @Test - public void test() { - new NodeInfoServiceTest(context).test(); - new DelegationServiceTest(context).test(); - } -} diff --git a/framework/src/test/java/org/tron/core/net/P2pEventHandlerImplTest.java b/framework/src/test/java/org/tron/core/net/P2pEventHandlerImplTest.java index e0c816a537a..99f115351ba 100644 --- a/framework/src/test/java/org/tron/core/net/P2pEventHandlerImplTest.java +++ b/framework/src/test/java/org/tron/core/net/P2pEventHandlerImplTest.java @@ -6,8 +6,10 @@ import java.util.ArrayList; import java.util.List; import org.junit.Assert; +import org.junit.BeforeClass; import org.junit.Test; import org.mockito.Mockito; +import org.tron.common.BaseTest; import org.tron.common.parameter.CommonParameter; import org.tron.common.utils.Sha256Hash; import org.tron.core.Constant; @@ -20,12 +22,15 @@ import org.tron.protos.Protocol; import org.tron.protos.Protocol.Inventory.InventoryType; -public class P2pEventHandlerImplTest { +public class P2pEventHandlerImplTest extends BaseTest { + + @BeforeClass + public static void init() throws Exception { + Args.setParam(new String[] {"--output-directory", dbPath(), "--debug"}, Constant.TEST_CONF); + } @Test public void testProcessInventoryMessage() throws Exception { - String[] a = new String[0]; - Args.setParam(a, Constant.TESTNET_CONF); CommonParameter parameter = CommonParameter.getInstance(); parameter.setMaxTps(10); @@ -114,9 +119,6 @@ public void testProcessInventoryMessage() throws Exception { @Test public void testUpdateLastInteractiveTime() throws Exception { - String[] a = new String[0]; - Args.setParam(a, Constant.TESTNET_CONF); - PeerConnection peer = new PeerConnection(); P2pEventHandlerImpl p2pEventHandler = new P2pEventHandlerImpl(); diff --git a/framework/src/test/java/org/tron/core/net/P2pRateLimiterTest.java b/framework/src/test/java/org/tron/core/net/P2pRateLimiterTest.java new file mode 100644 index 00000000000..8a1d9c52749 --- /dev/null +++ b/framework/src/test/java/org/tron/core/net/P2pRateLimiterTest.java @@ -0,0 +1,23 @@ +package org.tron.core.net; + +import static org.tron.core.net.message.MessageTypes.FETCH_INV_DATA; +import static org.tron.core.net.message.MessageTypes.SYNC_BLOCK_CHAIN; + +import org.junit.Assert; +import org.junit.Test; + +public class P2pRateLimiterTest { + @Test + public void test() { + P2pRateLimiter limiter = new P2pRateLimiter(); + limiter.register(SYNC_BLOCK_CHAIN.asByte(), 2); + limiter.acquire(SYNC_BLOCK_CHAIN.asByte()); + boolean ret = limiter.tryAcquire(SYNC_BLOCK_CHAIN.asByte()); + Assert.assertTrue(ret); + limiter.tryAcquire(SYNC_BLOCK_CHAIN.asByte()); + ret = limiter.tryAcquire(SYNC_BLOCK_CHAIN.asByte()); + Assert.assertFalse(ret); + ret = limiter.tryAcquire(FETCH_INV_DATA.asByte()); + Assert.assertTrue(ret); + } +} diff --git a/framework/src/test/java/org/tron/core/net/messagehandler/FetchInvDataMsgHandlerTest.java b/framework/src/test/java/org/tron/core/net/messagehandler/FetchInvDataMsgHandlerTest.java index 5fd6d6725ba..270002fffba 100644 --- a/framework/src/test/java/org/tron/core/net/messagehandler/FetchInvDataMsgHandlerTest.java +++ b/framework/src/test/java/org/tron/core/net/messagehandler/FetchInvDataMsgHandlerTest.java @@ -1,5 +1,7 @@ package org.tron.core.net.messagehandler; +import static org.tron.core.net.message.MessageTypes.FETCH_INV_DATA; + import com.google.common.cache.Cache; import com.google.common.cache.CacheBuilder; import java.lang.reflect.Field; @@ -13,6 +15,7 @@ import org.tron.common.utils.Sha256Hash; import org.tron.core.capsule.BlockCapsule; import org.tron.core.config.Parameter; +import org.tron.core.net.P2pRateLimiter; import org.tron.core.net.TronNetDelegate; import org.tron.core.net.message.adv.BlockMessage; import org.tron.core.net.message.adv.FetchInvDataMessage; @@ -55,12 +58,40 @@ public void testProcessMessage() throws Exception { Mockito.when(advService.getMessage(new Item(blockId, Protocol.Inventory.InventoryType.BLOCK))) .thenReturn(new BlockMessage(blockCapsule)); ReflectUtils.setFieldValue(fetchInvDataMsgHandler, "advService", advService); + P2pRateLimiter p2pRateLimiter = new P2pRateLimiter(); + p2pRateLimiter.register(FETCH_INV_DATA.asByte(), 2); + Mockito.when(peer.getP2pRateLimiter()).thenReturn(p2pRateLimiter); fetchInvDataMsgHandler.processMessage(peer, new FetchInvDataMessage(blockIds, Protocol.Inventory.InventoryType.BLOCK)); Assert.assertNotNull(syncBlockIdCache.getIfPresent(blockId)); } + @Test + public void testIsAdvInv() { + FetchInvDataMsgHandler fetchInvDataMsgHandler = new FetchInvDataMsgHandler(); + + List list = new LinkedList<>(); + list.add(Sha256Hash.ZERO_HASH); + FetchInvDataMessage msg = + new FetchInvDataMessage(list, Protocol.Inventory.InventoryType.TRX); + + boolean isAdv = fetchInvDataMsgHandler.isAdvInv(null, msg); + Assert.assertTrue(isAdv); + + PeerConnection peer = Mockito.mock(PeerConnection.class); + Cache advInvSpread = CacheBuilder.newBuilder().build(); + Mockito.when(peer.getAdvInvSpread()).thenReturn(advInvSpread); + + msg = new FetchInvDataMessage(list, Protocol.Inventory.InventoryType.BLOCK); + isAdv = fetchInvDataMsgHandler.isAdvInv(peer, msg); + Assert.assertTrue(!isAdv); + + advInvSpread.put(new Item(Sha256Hash.ZERO_HASH, Protocol.Inventory.InventoryType.BLOCK), 1L); + isAdv = fetchInvDataMsgHandler.isAdvInv(peer, msg); + Assert.assertTrue(isAdv); + } + @Test public void testSyncFetchCheck() { BlockCapsule.BlockId blockId = new BlockCapsule.BlockId(Sha256Hash.ZERO_HASH, 10000L); @@ -74,6 +105,9 @@ public void testSyncFetchCheck() { Cache advInvSpread = CacheBuilder.newBuilder().maximumSize(100) .expireAfterWrite(1, TimeUnit.HOURS).recordStats().build(); Mockito.when(peer.getAdvInvSpread()).thenReturn(advInvSpread); + P2pRateLimiter p2pRateLimiter = new P2pRateLimiter(); + p2pRateLimiter.register(FETCH_INV_DATA.asByte(), 2); + Mockito.when(peer.getP2pRateLimiter()).thenReturn(p2pRateLimiter); FetchInvDataMsgHandler fetchInvDataMsgHandler = new FetchInvDataMsgHandler(); @@ -93,4 +127,36 @@ public void testSyncFetchCheck() { Assert.assertEquals(e.getMessage(), "minBlockNum: 16000, blockNum: 10000"); } } + + @Test + public void testRateLimiter() { + BlockCapsule.BlockId blockId = new BlockCapsule.BlockId(Sha256Hash.ZERO_HASH, 10000L); + List blockIds = new LinkedList<>(); + for (int i = 0; i <= 100; i++) { + blockIds.add(blockId); + } + FetchInvDataMessage msg = + new FetchInvDataMessage(blockIds, Protocol.Inventory.InventoryType.BLOCK); + PeerConnection peer = Mockito.mock(PeerConnection.class); + Mockito.when(peer.isNeedSyncFromUs()).thenReturn(true); + Cache advInvSpread = CacheBuilder.newBuilder().maximumSize(100) + .expireAfterWrite(1, TimeUnit.HOURS).recordStats().build(); + Mockito.when(peer.getAdvInvSpread()).thenReturn(advInvSpread); + P2pRateLimiter p2pRateLimiter = new P2pRateLimiter(); + p2pRateLimiter.register(FETCH_INV_DATA.asByte(), 1); + p2pRateLimiter.acquire(FETCH_INV_DATA.asByte()); + Mockito.when(peer.getP2pRateLimiter()).thenReturn(p2pRateLimiter); + FetchInvDataMsgHandler fetchInvDataMsgHandler = new FetchInvDataMsgHandler(); + + try { + fetchInvDataMsgHandler.processMessage(peer, msg); + } catch (Exception e) { + Assert.assertEquals("fetch too many blocks, size:101", e.getMessage()); + } + try { + fetchInvDataMsgHandler.processMessage(peer, msg); + } catch (Exception e) { + Assert.assertTrue(e.getMessage().endsWith("rate limit")); + } + } } diff --git a/framework/src/test/java/org/tron/core/net/messagehandler/MessageHandlerTest.java b/framework/src/test/java/org/tron/core/net/messagehandler/MessageHandlerTest.java index 7ff29b54bb7..b5feb6a765a 100644 --- a/framework/src/test/java/org/tron/core/net/messagehandler/MessageHandlerTest.java +++ b/framework/src/test/java/org/tron/core/net/messagehandler/MessageHandlerTest.java @@ -3,13 +3,10 @@ import static org.mockito.Mockito.mock; import com.google.protobuf.ByteString; -import java.lang.reflect.Field; import java.net.InetSocketAddress; -import java.util.ArrayList; -import java.util.Collections; +import org.junit.After; import org.junit.AfterClass; import org.junit.Assert; -import org.junit.Before; import org.junit.BeforeClass; import org.junit.ClassRule; import org.junit.Test; @@ -63,14 +60,10 @@ public static void destroy() { context.destroy(); } - @Before + @After public void clearPeers() { - try { - Field field = PeerManager.class.getDeclaredField("peers"); - field.setAccessible(true); - field.set(PeerManager.class, Collections.synchronizedList(new ArrayList<>())); - } catch (NoSuchFieldException | IllegalAccessException e) { - //ignore + for (PeerConnection p : PeerManager.getPeers()) { + PeerManager.remove(p.getChannel()); } } diff --git a/framework/src/test/java/org/tron/core/net/messagehandler/PbftMsgHandlerTest.java b/framework/src/test/java/org/tron/core/net/messagehandler/PbftMsgHandlerTest.java index 8b9d1969cfc..4ae0e4e54b6 100644 --- a/framework/src/test/java/org/tron/core/net/messagehandler/PbftMsgHandlerTest.java +++ b/framework/src/test/java/org/tron/core/net/messagehandler/PbftMsgHandlerTest.java @@ -4,14 +4,11 @@ import com.google.protobuf.ByteString; import java.io.File; -import java.lang.reflect.Field; import java.net.InetSocketAddress; -import java.util.ArrayList; -import java.util.Collections; import org.bouncycastle.util.encoders.Hex; +import org.junit.After; import org.junit.AfterClass; import org.junit.Assert; -import org.junit.Before; import org.junit.BeforeClass; import org.junit.Test; import org.mockito.Mockito; @@ -65,14 +62,10 @@ public static void destroy() { FileUtil.deleteDir(new File(dbPath)); } - @Before + @After public void clearPeers() { - try { - Field field = PeerManager.class.getDeclaredField("peers"); - field.setAccessible(true); - field.set(PeerManager.class, Collections.synchronizedList(new ArrayList<>())); - } catch (NoSuchFieldException | IllegalAccessException e) { - //ignore + for (PeerConnection p : PeerManager.getPeers()) { + PeerManager.remove(p.getChannel()); } } diff --git a/framework/src/test/java/org/tron/core/net/messagehandler/SyncBlockChainMsgHandlerTest.java b/framework/src/test/java/org/tron/core/net/messagehandler/SyncBlockChainMsgHandlerTest.java index e654e1c9cc2..0d66208c1bf 100644 --- a/framework/src/test/java/org/tron/core/net/messagehandler/SyncBlockChainMsgHandlerTest.java +++ b/framework/src/test/java/org/tron/core/net/messagehandler/SyncBlockChainMsgHandlerTest.java @@ -1,15 +1,17 @@ package org.tron.core.net.messagehandler; +import java.io.IOException; import java.lang.reflect.Field; import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; import java.net.InetSocketAddress; import java.util.ArrayList; import java.util.List; -import org.junit.After; +import org.junit.AfterClass; import org.junit.Assert; import org.junit.Before; -import org.junit.Rule; +import org.junit.BeforeClass; +import org.junit.ClassRule; import org.junit.Test; import org.junit.rules.TemporaryFolder; import org.tron.common.application.TronApplicationContext; @@ -22,21 +24,34 @@ import org.tron.core.net.message.sync.BlockInventoryMessage; import org.tron.core.net.message.sync.SyncBlockChainMessage; import org.tron.core.net.peer.PeerConnection; +import org.tron.core.net.peer.PeerManager; import org.tron.p2p.connection.Channel; public class SyncBlockChainMsgHandlerTest { - private TronApplicationContext context; + private static TronApplicationContext context; private SyncBlockChainMsgHandler handler; private PeerConnection peer; - @Rule - public TemporaryFolder temporaryFolder = new TemporaryFolder(); + @ClassRule + public static final TemporaryFolder temporaryFolder = new TemporaryFolder(); + + public static String dbPath() { + try { + return temporaryFolder.newFolder().toString(); + } catch (IOException e) { + Assert.fail("create temp folder failed"); + } + return null; + } + + @BeforeClass + public static void before() { + Args.setParam(new String[] {"--output-directory", dbPath()}, Constant.TEST_CONF); + context = new TronApplicationContext(DefaultConfig.class); + } @Before public void init() throws Exception { - Args.setParam(new String[]{"--output-directory", - temporaryFolder.newFolder().toString(), "--debug"}, Constant.TEST_CONF); - context = new TronApplicationContext(DefaultConfig.class); handler = context.getBean(SyncBlockChainMsgHandler.class); peer = context.getBean(PeerConnection.class); Channel c1 = new Channel(); @@ -55,6 +70,7 @@ public void init() throws Exception { @Test public void testProcessMessage() throws Exception { try { + peer.setRemainNum(1); handler.processMessage(peer, new SyncBlockChainMessage(new ArrayList<>())); } catch (P2pException e) { Assert.assertEquals("SyncBlockChain blockIds is empty", e.getMessage()); @@ -64,13 +80,17 @@ public void testProcessMessage() throws Exception { blockIds.add(new BlockCapsule.BlockId()); SyncBlockChainMessage message = new SyncBlockChainMessage(blockIds); Method method = handler.getClass().getDeclaredMethod( - "check", PeerConnection.class, SyncBlockChainMessage.class); + "check", PeerConnection.class, SyncBlockChainMessage.class); method.setAccessible(true); - boolean f = (boolean)method.invoke(handler, peer, message); + boolean f = (boolean) method.invoke(handler, peer, message); Assert.assertNotNull(message.getAnswerMessage()); Assert.assertNotNull(message.toString()); Assert.assertNotNull(((BlockInventoryMessage) message).getAnswerMessage()); Assert.assertFalse(f); + method.invoke(handler, peer, message); + method.invoke(handler, peer, message); + f = (boolean) method.invoke(handler, peer, message); + Assert.assertFalse(f); Method method1 = handler.getClass().getDeclaredMethod( "getLostBlockIds", List.class, BlockId.class); @@ -88,8 +108,12 @@ public void testProcessMessage() throws Exception { Assert.assertEquals(1, list.size()); } - @After - public void destroy() { + @AfterClass + public static void destroy() { + for (PeerConnection p : PeerManager.getPeers()) { + PeerManager.remove(p.getChannel()); + } + context.destroy(); Args.clearParam(); } diff --git a/framework/src/test/java/org/tron/core/net/peer/PeerConnectionTest.java b/framework/src/test/java/org/tron/core/net/peer/PeerConnectionTest.java index 5a67cd8f609..649e5eb0875 100644 --- a/framework/src/test/java/org/tron/core/net/peer/PeerConnectionTest.java +++ b/framework/src/test/java/org/tron/core/net/peer/PeerConnectionTest.java @@ -7,14 +7,18 @@ import java.util.LinkedList; import java.util.List; +import org.junit.AfterClass; import org.junit.Assert; +import org.junit.BeforeClass; import org.junit.Test; import org.mockito.Mockito; import org.tron.common.overlay.message.Message; +import org.tron.common.parameter.CommonParameter; import org.tron.common.utils.Pair; import org.tron.common.utils.ReflectUtils; import org.tron.common.utils.Sha256Hash; import org.tron.core.capsule.BlockCapsule; +import org.tron.core.config.args.Args; import org.tron.core.net.message.adv.InventoryMessage; import org.tron.core.net.message.handshake.HelloMessage; import org.tron.core.net.message.keepalive.PingMessage; @@ -26,6 +30,18 @@ public class PeerConnectionTest { + @BeforeClass + public static void initArgs() { + CommonParameter.getInstance().setRateLimiterSyncBlockChain(10); + CommonParameter.getInstance().setRateLimiterFetchInvData(10); + CommonParameter.getInstance().setRateLimiterDisconnect(10); + } + + @AfterClass + public static void destroy() { + Args.clearParam(); + } + @Test public void testVariableDefaultValue() { PeerConnection peerConnection = new PeerConnection(); diff --git a/framework/src/test/java/org/tron/core/net/peer/PeerManagerTest.java b/framework/src/test/java/org/tron/core/net/peer/PeerManagerTest.java index b8c03de1029..ee409a8ab04 100644 --- a/framework/src/test/java/org/tron/core/net/peer/PeerManagerTest.java +++ b/framework/src/test/java/org/tron/core/net/peer/PeerManagerTest.java @@ -8,16 +8,40 @@ import java.util.Collections; import java.util.List; +import org.junit.After; +import org.junit.AfterClass; import org.junit.Assert; +import org.junit.BeforeClass; import org.junit.Test; import org.mockito.Mockito; import org.springframework.context.ApplicationContext; +import org.tron.common.parameter.CommonParameter; import org.tron.common.utils.ReflectUtils; +import org.tron.core.config.args.Args; import org.tron.p2p.connection.Channel; public class PeerManagerTest { List relayNodes = new ArrayList<>(); + @BeforeClass + public static void initArgs() { + CommonParameter.getInstance().setRateLimiterSyncBlockChain(10); + CommonParameter.getInstance().setRateLimiterFetchInvData(10); + CommonParameter.getInstance().setRateLimiterDisconnect(10); + } + + @AfterClass + public static void destroy() { + Args.clearParam(); + } + + @After + public void clearPeers() { + for (PeerConnection p : PeerManager.getPeers()) { + PeerManager.remove(p.getChannel()); + } + } + @Test public void testAdd() throws Exception { Field field = PeerManager.class.getDeclaredField("peers"); diff --git a/framework/src/test/java/org/tron/core/net/service/nodepersist/NodePersistServiceTest.java b/framework/src/test/java/org/tron/core/net/service/nodepersist/NodePersistServiceTest.java new file mode 100644 index 00000000000..cd80a6b78f0 --- /dev/null +++ b/framework/src/test/java/org/tron/core/net/service/nodepersist/NodePersistServiceTest.java @@ -0,0 +1,40 @@ +package org.tron.core.net.service.nodepersist; + +import java.util.ArrayList; +import java.util.List; +import javax.annotation.Resource; +import org.junit.Assert; +import org.junit.BeforeClass; +import org.junit.Test; +import org.tron.common.BaseTest; +import org.tron.common.utils.JsonUtil; +import org.tron.core.Constant; +import org.tron.core.capsule.BytesCapsule; +import org.tron.core.config.args.Args; + + +public class NodePersistServiceTest extends BaseTest { + + @Resource + private NodePersistService nodePersistService; + + @BeforeClass + public static void init() { + Args.setParam(new String[] {"--output-directory", dbPath(), "--debug"}, + Constant.TEST_CONF); + } + + @Test + public void testDbRead() { + byte[] DB_KEY_PEERS = "peers".getBytes(); + DBNode dbNode = new DBNode("localhost", 3306); + List dbNodeList = new ArrayList<>(); + dbNodeList.add(dbNode); + DBNodes dbNodes = new DBNodes(); + dbNodes.setNodes(dbNodeList); + chainBaseManager.getCommonStore().put(DB_KEY_PEERS, new BytesCapsule( + JsonUtil.obj2Json(dbNodes).getBytes())); + + Assert.assertEquals(1, nodePersistService.dbRead().size()); + } +} diff --git a/framework/src/test/java/org/tron/core/net/services/AdvServiceTest.java b/framework/src/test/java/org/tron/core/net/services/AdvServiceTest.java index 04a6315f522..cfaa44574e3 100644 --- a/framework/src/test/java/org/tron/core/net/services/AdvServiceTest.java +++ b/framework/src/test/java/org/tron/core/net/services/AdvServiceTest.java @@ -53,6 +53,9 @@ public static void init() throws IOException { @AfterClass public static void after() { + for (PeerConnection p : PeerManager.getPeers()) { + PeerManager.remove(p.getChannel()); + } Args.clearParam(); context.destroy(); } diff --git a/framework/src/test/java/org/tron/core/net/services/EffectiveCheckServiceTest.java b/framework/src/test/java/org/tron/core/net/services/EffectiveCheckServiceTest.java index 9104b087d26..da7f714d096 100644 --- a/framework/src/test/java/org/tron/core/net/services/EffectiveCheckServiceTest.java +++ b/framework/src/test/java/org/tron/core/net/services/EffectiveCheckServiceTest.java @@ -1,48 +1,35 @@ package org.tron.core.net.services; -import java.io.IOException; import java.lang.reflect.Method; import java.net.InetSocketAddress; -import org.junit.After; +import javax.annotation.Resource; import org.junit.Assert; -import org.junit.Before; -import org.junit.Rule; +import org.junit.BeforeClass; import org.junit.Test; -import org.junit.rules.TemporaryFolder; -import org.tron.common.application.TronApplicationContext; +import org.tron.common.BaseTest; +import org.tron.common.utils.PublicMethod; import org.tron.common.utils.ReflectUtils; import org.tron.core.Constant; -import org.tron.core.config.DefaultConfig; import org.tron.core.config.args.Args; import org.tron.core.net.TronNetService; import org.tron.core.net.service.effective.EffectiveCheckService; import org.tron.p2p.P2pConfig; -public class EffectiveCheckServiceTest { +public class EffectiveCheckServiceTest extends BaseTest { - protected TronApplicationContext context; + @Resource private EffectiveCheckService service; + @Resource + private TronNetService tronNetService; - @Rule - public final TemporaryFolder temporaryFolder = new TemporaryFolder(); - - @Before - public void init() throws IOException { - Args.setParam(new String[] {"--output-directory", - temporaryFolder.newFolder().toString(), "--debug"}, Constant.TEST_CONF); - context = new TronApplicationContext(DefaultConfig.class); - service = context.getBean(EffectiveCheckService.class); - } - - @After - public void destroy() { - Args.clearParam(); - context.destroy(); + @BeforeClass + public static void init() { + Args.setParam(new String[] {"--output-directory", dbPath(), "--debug"}, + Constant.TEST_CONF); } @Test public void testNoIpv4() throws Exception { - TronNetService tronNetService = context.getBean(TronNetService.class); Method privateMethod = tronNetService.getClass() .getDeclaredMethod("updateConfig", P2pConfig.class); privateMethod.setAccessible(true); @@ -54,10 +41,10 @@ public void testNoIpv4() throws Exception { @Test public void testFind() { - TronNetService tronNetService = context.getBean(TronNetService.class); + int port = PublicMethod.chooseRandomPort(); P2pConfig p2pConfig = new P2pConfig(); p2pConfig.setIp("127.0.0.1"); - p2pConfig.setPort(34567); + p2pConfig.setPort(port); ReflectUtils.setFieldValue(tronNetService, "p2pConfig", p2pConfig); TronNetService.getP2pService().start(p2pConfig); @@ -65,7 +52,7 @@ public void testFind() { Assert.assertNull(service.getCur()); ReflectUtils.invokeMethod(service, "resetCount"); - InetSocketAddress cur = new InetSocketAddress("192.168.0.1", 34567); + InetSocketAddress cur = new InetSocketAddress("192.168.0.1", port); service.setCur(cur); service.onDisconnect(cur); } diff --git a/framework/src/test/java/org/tron/core/net/services/HandShakeServiceTest.java b/framework/src/test/java/org/tron/core/net/services/HandShakeServiceTest.java index f4fabce5d64..0d8a50c8a92 100644 --- a/framework/src/test/java/org/tron/core/net/services/HandShakeServiceTest.java +++ b/framework/src/test/java/org/tron/core/net/services/HandShakeServiceTest.java @@ -4,16 +4,13 @@ import static org.tron.core.net.message.handshake.HelloMessage.getEndpointFromNode; import com.google.protobuf.ByteString; -import java.lang.reflect.Field; import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; import java.net.InetSocketAddress; -import java.util.ArrayList; -import java.util.Collections; import java.util.Random; +import org.junit.After; import org.junit.AfterClass; import org.junit.Assert; -import org.junit.Before; import org.junit.BeforeClass; import org.junit.ClassRule; import org.junit.Test; @@ -21,7 +18,6 @@ import org.mockito.Mockito; import org.springframework.context.ApplicationContext; import org.tron.common.application.TronApplicationContext; -import org.tron.common.utils.ByteArray; import org.tron.common.utils.ReflectUtils; import org.tron.common.utils.Sha256Hash; import org.tron.core.ChainBaseManager; @@ -34,6 +30,7 @@ import org.tron.core.net.message.handshake.HelloMessage; import org.tron.core.net.peer.PeerConnection; import org.tron.core.net.peer.PeerManager; +import org.tron.core.net.service.handshake.HandshakeService; import org.tron.p2p.P2pConfig; import org.tron.p2p.base.Parameter; import org.tron.p2p.connection.Channel; @@ -72,14 +69,10 @@ public static void destroy() { context.destroy(); } - @Before + @After public void clearPeers() { - try { - Field field = PeerManager.class.getDeclaredField("peers"); - field.setAccessible(true); - field.set(PeerManager.class, Collections.synchronizedList(new ArrayList<>())); - } catch (NoSuchFieldException | IllegalAccessException e) { - //ignore + for (PeerConnection p : PeerManager.getPeers()) { + PeerManager.remove(p.getChannel()); } } @@ -101,6 +94,7 @@ public void testOkHelloMessage() Node node = new Node(NetUtil.getNodeId(), a1.getAddress().getHostAddress(), null, a1.getPort()); HelloMessage helloMessage = new HelloMessage(node, System.currentTimeMillis(), ChainBaseManager.getChainBaseManager()); + Assert.assertNotNull(helloMessage.toString()); Assert.assertEquals(Version.getVersion(), new String(helloMessage.getHelloMessage().getCodeVersion().toByteArray())); @@ -126,13 +120,27 @@ public void testInvalidHelloMessage() { //block hash is empty try { BlockCapsule.BlockId hid = ChainBaseManager.getChainBaseManager().getHeadBlockId(); - Protocol.HelloMessage.BlockId hBlockId = Protocol.HelloMessage.BlockId.newBuilder() - .setHash(ByteString.copyFrom(new byte[0])) + Protocol.HelloMessage.BlockId okBlockId = Protocol.HelloMessage.BlockId.newBuilder() + .setHash(ByteString.copyFrom(new byte[32])) + .setNumber(hid.getNum()) + .build(); + Protocol.HelloMessage.BlockId invalidBlockId = Protocol.HelloMessage.BlockId.newBuilder() + .setHash(ByteString.copyFrom(new byte[31])) .setNumber(hid.getNum()) .build(); - builder.setHeadBlockId(hBlockId); + builder.setHeadBlockId(invalidBlockId); HelloMessage helloMessage = new HelloMessage(builder.build().toByteArray()); - Assert.assertTrue(!helloMessage.valid()); + Assert.assertFalse(helloMessage.valid()); + + builder.setHeadBlockId(okBlockId); + builder.setGenesisBlockId(invalidBlockId); + HelloMessage helloMessage2 = new HelloMessage(builder.build().toByteArray()); + Assert.assertFalse(helloMessage2.valid()); + + builder.setGenesisBlockId(okBlockId); + builder.setSolidBlockId(invalidBlockId); + HelloMessage helloMessage3 = new HelloMessage(builder.build().toByteArray()); + Assert.assertFalse(helloMessage3.valid()); } catch (Exception e) { Assert.fail(); } @@ -214,7 +222,7 @@ public void testLowAndGenesisBlockNum() throws NoSuchMethodException { Node node2 = new Node(NetUtil.getNodeId(), a1.getAddress().getHostAddress(), null, 10002); - //lowestBlockNum > headBlockNum + //peer's lowestBlockNum > my headBlockNum => peer is light, LIGHT_NODE_SYNC_FAIL Protocol.HelloMessage.Builder builder = getHelloMessageBuilder(node2, System.currentTimeMillis(), ChainBaseManager.getChainBaseManager()); @@ -226,7 +234,7 @@ public void testLowAndGenesisBlockNum() throws NoSuchMethodException { Assert.fail(); } - //genesisBlock is not equal + //genesisBlock is not equal => INCOMPATIBLE_CHAIN builder = getHelloMessageBuilder(node2, System.currentTimeMillis(), ChainBaseManager.getChainBaseManager()); BlockCapsule.BlockId gid = ChainBaseManager.getChainBaseManager().getGenesisBlockId(); @@ -242,9 +250,11 @@ public void testLowAndGenesisBlockNum() throws NoSuchMethodException { Assert.fail(); } - //solidityBlock <= us, but not contained + // peer's solidityBlock <= my solidityBlock, but not contained + // and my lowestBlockNum <= peer's solidityBlock => FORKED builder = getHelloMessageBuilder(node2, System.currentTimeMillis(), ChainBaseManager.getChainBaseManager()); + BlockCapsule.BlockId sid = ChainBaseManager.getChainBaseManager().getSolidBlockId(); Random gen = new Random(); @@ -262,6 +272,46 @@ public void testLowAndGenesisBlockNum() throws NoSuchMethodException { } catch (Exception e) { Assert.fail(); } + + // peer's solidityBlock <= my solidityBlock, but not contained + // and my lowestBlockNum > peer's solidityBlock => i am light, LIGHT_NODE_SYNC_FAIL + ChainBaseManager.getChainBaseManager().setLowestBlockNum(2); + try { + HelloMessage helloMessage = new HelloMessage(builder.build().toByteArray()); + method.invoke(p2pEventHandler, peer, helloMessage.getSendBytes()); + } catch (Exception e) { + Assert.fail(); + } + } + + @Test + public void testProcessHelloMessage() { + InetSocketAddress a1 = new InetSocketAddress("127.0.0.1", 10001); + Channel c1 = mock(Channel.class); + Mockito.when(c1.getInetSocketAddress()).thenReturn(a1); + Mockito.when(c1.getInetAddress()).thenReturn(a1.getAddress()); + PeerManager.add(ctx, c1); + PeerConnection p = PeerManager.getPeers().get(0); + + try { + Node node = new Node(NetUtil.getNodeId(), a1.getAddress().getHostAddress(), + null, a1.getPort()); + Protocol.HelloMessage.Builder builder = + getHelloMessageBuilder(node, System.currentTimeMillis(), + ChainBaseManager.getChainBaseManager()); + BlockCapsule.BlockId hid = ChainBaseManager.getChainBaseManager().getHeadBlockId(); + Protocol.HelloMessage.BlockId invalidBlockId = Protocol.HelloMessage.BlockId.newBuilder() + .setHash(ByteString.copyFrom(new byte[31])) + .setNumber(hid.getNum()) + .build(); + builder.setHeadBlockId(invalidBlockId); + + HelloMessage helloMessage = new HelloMessage(builder.build().toByteArray()); + HandshakeService handshakeService = new HandshakeService(); + handshakeService.processHelloMessage(p, helloMessage); + } catch (Exception e) { + Assert.fail(); + } } private Protocol.HelloMessage.Builder getHelloMessageBuilder(Node from, long timestamp, diff --git a/framework/src/test/java/org/tron/core/net/services/RelayServiceTest.java b/framework/src/test/java/org/tron/core/net/services/RelayServiceTest.java index 5e22e538e80..63028001249 100644 --- a/framework/src/test/java/org/tron/core/net/services/RelayServiceTest.java +++ b/framework/src/test/java/org/tron/core/net/services/RelayServiceTest.java @@ -1,10 +1,12 @@ package org.tron.core.net.services; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.doNothing; import static org.mockito.Mockito.mock; -import com.google.common.collect.Lists; import com.google.protobuf.ByteString; import java.lang.reflect.Field; +import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; import java.net.InetSocketAddress; import java.util.ArrayList; @@ -12,28 +14,36 @@ import java.util.List; import java.util.Set; import javax.annotation.Resource; - import lombok.extern.slf4j.Slf4j; import org.bouncycastle.util.encoders.Hex; +import org.junit.After; import org.junit.Assert; import org.junit.BeforeClass; import org.junit.Test; import org.mockito.Mockito; import org.springframework.context.ApplicationContext; import org.tron.common.BaseTest; +import org.tron.common.crypto.SignInterface; +import org.tron.common.crypto.SignUtils; +import org.tron.common.parameter.CommonParameter; +import org.tron.common.utils.ByteArray; import org.tron.common.utils.ReflectUtils; +import org.tron.common.utils.Sha256Hash; import org.tron.core.ChainBaseManager; import org.tron.core.Constant; import org.tron.core.capsule.BlockCapsule; import org.tron.core.capsule.WitnessCapsule; import org.tron.core.config.args.Args; import org.tron.core.net.P2pEventHandlerImpl; +import org.tron.core.net.TronNetDelegate; +import org.tron.core.net.TronNetService; import org.tron.core.net.message.adv.BlockMessage; import org.tron.core.net.message.handshake.HelloMessage; import org.tron.core.net.peer.Item; import org.tron.core.net.peer.PeerConnection; import org.tron.core.net.peer.PeerManager; import org.tron.core.net.service.relay.RelayService; +import org.tron.p2p.P2pConfig; import org.tron.p2p.connection.Channel; import org.tron.p2p.discover.Node; import org.tron.p2p.utils.NetUtil; @@ -44,11 +54,13 @@ public class RelayServiceTest extends BaseTest { @Resource private RelayService service; - @Resource - private PeerConnection peer; + @Resource private P2pEventHandlerImpl p2pEventHandler; + @Resource + private TronNetService tronNetService; + /** * init context. */ @@ -58,6 +70,11 @@ public static void init() { Constant.TEST_CONF); } + @After + public void clearPeers() { + closePeer(); + } + @Test public void test() throws Exception { initWitness(); @@ -67,8 +84,12 @@ public void test() throws Exception { } private void initWitness() { - byte[] key = Hex.decode("A04711BF7AFBDF44557DEFBDF4C4E7AA6138C6331F"); + // key: 0154435f065a57fec6af1e12eaa2fa600030639448d7809f4c65bdcf8baed7e5 + // Hex: A08A8D690BF36806C36A7DAE3AF796643C1AA9CC01 + // Base58: 27bi7CD8d94AgXY3XFS9A9vx78Si5MqrECz + byte[] key = Hex.decode("A08A8D690BF36806C36A7DAE3AF796643C1AA9CC01");//exist already WitnessCapsule witnessCapsule = chainBaseManager.getWitnessStore().get(key); + System.out.println(witnessCapsule.getInstance()); witnessCapsule.setVoteCount(1000); chainBaseManager.getWitnessStore().put(key, witnessCapsule); List list = new ArrayList<>(); @@ -83,7 +104,7 @@ public void testGetNextWitnesses() throws Exception { "getNextWitnesses", ByteString.class, Integer.class); method.setAccessible(true); Set s1 = (Set) method.invoke( - service, getFromHexString("A04711BF7AFBDF44557DEFBDF4C4E7AA6138C6331F"), 3); + service, getFromHexString("A08A8D690BF36806C36A7DAE3AF796643C1AA9CC01"), 3); Assert.assertEquals(3, s1.size()); assertContains(s1, "A0299F3DB80A24B20A254B89CE639D59132F157F13"); assertContains(s1, "A0807337F180B62A77576377C1D0C9C24DF5C0DD62"); @@ -93,35 +114,51 @@ public void testGetNextWitnesses() throws Exception { service, getFromHexString("A0FAB5FBF6AFB681E4E37E9D33BDDB7E923D6132E5"), 3); Assert.assertEquals(3, s2.size()); assertContains(s2, "A014EEBE4D30A6ACB505C8B00B218BDC4733433C68"); - assertContains(s2, "A04711BF7AFBDF44557DEFBDF4C4E7AA6138C6331F"); + assertContains(s2, "A08A8D690BF36806C36A7DAE3AF796643C1AA9CC01"); assertContains(s2, "A0299F3DB80A24B20A254B89CE639D59132F157F13"); Set s3 = (Set) method.invoke( - service, getFromHexString("A04711BF7AFBDF44557DEFBDF4C4E7AA6138C6331F"), 1); + service, getFromHexString("A08A8D690BF36806C36A7DAE3AF796643C1AA9CC01"), 1); Assert.assertEquals(1, s3.size()); assertContains(s3, "A0299F3DB80A24B20A254B89CE639D59132F157F13"); } private void testBroadcast() { try { + PeerConnection peer = new PeerConnection(); + InetSocketAddress a1 = new InetSocketAddress("127.0.0.2", 10001); + Channel c1 = mock(Channel.class); + Mockito.when(c1.getInetSocketAddress()).thenReturn(a1); + Mockito.when(c1.getInetAddress()).thenReturn(a1.getAddress()); + doNothing().when(c1).send((byte[]) any()); + + peer.setChannel(c1); peer.setAddress(getFromHexString("A0299F3DB80A24B20A254B89CE639D59132F157F13")); peer.setNeedSyncFromPeer(false); peer.setNeedSyncFromUs(false); - List peers = Lists.newArrayList(); + List peers = new ArrayList<>(); peers.add(peer); - ReflectUtils.setFieldValue(p2pEventHandler, "activePeers", peers); + + TronNetDelegate tronNetDelegate = Mockito.mock(TronNetDelegate.class); + Mockito.doReturn(peers).when(tronNetDelegate).getActivePeer(); + + Field field = service.getClass().getDeclaredField("tronNetDelegate"); + field.setAccessible(true); + field.set(service, tronNetDelegate); + BlockCapsule blockCapsule = new BlockCapsule(chainBaseManager.getHeadBlockNum() + 1, chainBaseManager.getHeadBlockId(), - 0, getFromHexString("A04711BF7AFBDF44557DEFBDF4C4E7AA6138C6331F")); + 0, getFromHexString("A08A8D690BF36806C36A7DAE3AF796643C1AA9CC01")); BlockMessage msg = new BlockMessage(blockCapsule); service.broadcast(msg); Item item = new Item(blockCapsule.getBlockId(), Protocol.Inventory.InventoryType.BLOCK); Assert.assertEquals(1, peer.getAdvInvSpread().size()); Assert.assertNotNull(peer.getAdvInvSpread().getIfPresent(item)); peer.getChannel().close(); - } catch (NullPointerException e) { - System.out.println(e); + } catch (Exception e) { + logger.info("", e); + assert false; } } @@ -135,20 +172,32 @@ private ByteString getFromHexString(String s) { } private void testCheckHelloMessage() { - ByteString address = getFromHexString("A04711BF7AFBDF44557DEFBDF4C4E7AA6138C6331F"); + String key = "0154435f065a57fec6af1e12eaa2fa600030639448d7809f4c65bdcf8baed7e5"; + ByteString address = getFromHexString("A08A8D690BF36806C36A7DAE3AF796643C1AA9CC01"); InetSocketAddress a1 = new InetSocketAddress("127.0.0.1", 10001); Node node = new Node(NetUtil.getNodeId(), a1.getAddress().getHostAddress(), null, a1.getPort()); + + SignInterface cryptoEngine = SignUtils.fromPrivate(ByteArray.fromHexString(key), + Args.getInstance().isECKeyCryptoEngine()); HelloMessage helloMessage = new HelloMessage(node, System.currentTimeMillis(), ChainBaseManager.getChainBaseManager()); + ByteString sig = ByteString.copyFrom(cryptoEngine.Base64toBytes(cryptoEngine + .signHash(Sha256Hash.of(CommonParameter.getInstance() + .isECKeyCryptoEngine(), ByteArray.fromLong(helloMessage + .getTimestamp())).getBytes()))); helloMessage.setHelloMessage(helloMessage.getHelloMessage().toBuilder() - .setAddress(address).build()); + .setAddress(address) + .setSignature(sig) + .build()); + Channel c1 = mock(Channel.class); Mockito.when(c1.getInetSocketAddress()).thenReturn(a1); Mockito.when(c1.getInetAddress()).thenReturn(a1.getAddress()); Channel c2 = mock(Channel.class); Mockito.when(c2.getInetSocketAddress()).thenReturn(a1); Mockito.when(c2.getInetAddress()).thenReturn(a1.getAddress()); + Args.getInstance().fastForward = true; ApplicationContext ctx = (ApplicationContext) ReflectUtils.getFieldObject(p2pEventHandler, "ctx"); @@ -158,14 +207,51 @@ private void testCheckHelloMessage() { PeerConnection peer2 = PeerManager.add(ctx, c2); assert peer2 != null; peer2.setAddress(address); + + ReflectUtils.setFieldValue(tronNetService, "p2pConfig", new P2pConfig()); + try { Field field = service.getClass().getDeclaredField("witnessScheduleStore"); field.setAccessible(true); field.set(service, chainBaseManager.getWitnessScheduleStore()); + + Field field2 = service.getClass().getDeclaredField("manager"); + field2.setAccessible(true); + field2.set(service, dbManager); + boolean res = service.checkHelloMessage(helloMessage, c1); - Assert.assertFalse(res); + Assert.assertTrue(res); } catch (Exception e) { - logger.info("{}", e.getMessage()); + logger.info("", e); + assert false; + } + } + + @Test + public void testNullWitnessAddress() { + try { + Class clazz = service.getClass(); + + Field keySizeField = clazz.getDeclaredField("keySize"); + keySizeField.setAccessible(true); + keySizeField.set(service, 0); + + Field witnessAddressField = clazz.getDeclaredField("witnessAddress"); + witnessAddressField.setAccessible(true); + witnessAddressField.set(service, null); + + Method isActiveWitnessMethod = clazz.getDeclaredMethod("isActiveWitness"); + isActiveWitnessMethod.setAccessible(true); + + Boolean result = (Boolean) isActiveWitnessMethod.invoke(service); + Assert.assertNotEquals(Boolean.TRUE, result); + + witnessAddressField.set(service, ByteString.copyFrom(new byte[21])); + result = (Boolean) isActiveWitnessMethod.invoke(service); + Assert.assertNotEquals(Boolean.TRUE, result); + } catch (NoSuchMethodException | NoSuchFieldException + | IllegalAccessException | InvocationTargetException e) { + Assert.fail("Reflection invocation failed: " + e.getMessage()); } } -} \ No newline at end of file +} diff --git a/framework/src/test/java/org/tron/core/net/services/ResilienceServiceTest.java b/framework/src/test/java/org/tron/core/net/services/ResilienceServiceTest.java index ce723e5c991..bb1a0607796 100644 --- a/framework/src/test/java/org/tron/core/net/services/ResilienceServiceTest.java +++ b/framework/src/test/java/org/tron/core/net/services/ResilienceServiceTest.java @@ -8,49 +8,50 @@ import java.net.InetSocketAddress; import java.util.HashSet; import java.util.Set; +import javax.annotation.Resource; import org.junit.After; import org.junit.Assert; -import org.junit.Before; -import org.junit.Rule; +import org.junit.BeforeClass; import org.junit.Test; -import org.junit.rules.TemporaryFolder; import org.mockito.Mockito; -import org.tron.common.application.TronApplicationContext; +import org.springframework.context.ApplicationContext; +import org.tron.common.BaseTest; import org.tron.common.utils.ReflectUtils; -import org.tron.core.ChainBaseManager; import org.tron.core.Constant; -import org.tron.core.config.DefaultConfig; import org.tron.core.config.args.Args; +import org.tron.core.net.P2pEventHandlerImpl; import org.tron.core.net.peer.PeerConnection; import org.tron.core.net.peer.PeerManager; import org.tron.core.net.service.effective.ResilienceService; import org.tron.p2p.connection.Channel; -public class ResilienceServiceTest { +public class ResilienceServiceTest extends BaseTest { - protected TronApplicationContext context; + @Resource private ResilienceService service; - private ChainBaseManager chainBaseManager; - - @Rule - public final TemporaryFolder temporaryFolder = new TemporaryFolder(); - - @Before - public void init() throws IOException { - Args.setParam(new String[] {"--output-directory", - temporaryFolder.newFolder().toString(), "--debug"}, Constant.TEST_CONF); - context = new TronApplicationContext(DefaultConfig.class); - chainBaseManager = context.getBean(ChainBaseManager.class); - service = context.getBean(ResilienceService.class); + + @Resource + private P2pEventHandlerImpl p2pEventHandler; + + + @BeforeClass + public static void init() throws IOException { + Args.setParam(new String[] {"--output-directory", dbPath(), "--debug"}, Constant.TEST_CONF); + } + + @After + public void clearPeers() { + closePeer(); } @Test public void testDisconnectRandom() { int maxConnection = 30; Assert.assertEquals(maxConnection, Args.getInstance().getMaxConnections()); - clearPeers(); Assert.assertEquals(0, PeerManager.getPeers().size()); + ApplicationContext ctx = (ApplicationContext) ReflectUtils.getFieldObject(p2pEventHandler, + "ctx"); for (int i = 0; i < maxConnection + 1; i++) { InetSocketAddress inetSocketAddress = new InetSocketAddress("201.0.0." + i, 10001); Channel c1 = spy(Channel.class); @@ -59,7 +60,7 @@ public void testDisconnectRandom() { ReflectUtils.setFieldValue(c1, "ctx", spy(ChannelHandlerContext.class)); Mockito.doNothing().when(c1).send((byte[]) any()); - PeerManager.add(context, c1); + PeerManager.add(ctx, c1); } for (PeerConnection peer : PeerManager.getPeers() .subList(0, ResilienceService.minBroadcastPeerSize)) { @@ -99,9 +100,10 @@ public void testDisconnectRandom() { public void testDisconnectLan() { int minConnection = 8; Assert.assertEquals(minConnection, Args.getInstance().getMinConnections()); - clearPeers(); Assert.assertEquals(0, PeerManager.getPeers().size()); + ApplicationContext ctx = (ApplicationContext) ReflectUtils.getFieldObject(p2pEventHandler, + "ctx"); for (int i = 0; i < 9; i++) { InetSocketAddress inetSocketAddress = new InetSocketAddress("201.0.0." + i, 10001); Channel c1 = spy(Channel.class); @@ -111,7 +113,7 @@ public void testDisconnectLan() { ReflectUtils.setFieldValue(c1, "ctx", spy(ChannelHandlerContext.class)); Mockito.doNothing().when(c1).send((byte[]) any()); - PeerManager.add(context, c1); + PeerManager.add(ctx, c1); } for (PeerConnection peer : PeerManager.getPeers()) { peer.setNeedSyncFromPeer(false); @@ -154,10 +156,11 @@ public void testDisconnectLan() { public void testDisconnectIsolated2() { int maxConnection = 30; Assert.assertEquals(maxConnection, Args.getInstance().getMaxConnections()); - clearPeers(); Assert.assertEquals(0, PeerManager.getPeers().size()); int addSize = (int) (maxConnection * ResilienceService.retentionPercent) + 2; //26 + ApplicationContext ctx = (ApplicationContext) ReflectUtils.getFieldObject(p2pEventHandler, + "ctx"); for (int i = 0; i < addSize; i++) { InetSocketAddress inetSocketAddress = new InetSocketAddress("201.0.0." + i, 10001); Channel c1 = spy(Channel.class); @@ -168,7 +171,7 @@ public void testDisconnectIsolated2() { ReflectUtils.setFieldValue(c1, "ctx", spy(ChannelHandlerContext.class)); Mockito.doNothing().when(c1).send((byte[]) any()); - PeerManager.add(context, c1); + PeerManager.add(ctx, c1); } PeerManager.getPeers().get(10).setNeedSyncFromUs(false); PeerManager.getPeers().get(10).setNeedSyncFromPeer(false); @@ -190,16 +193,4 @@ public void testDisconnectIsolated2() { Assert.assertEquals((int) (maxConnection * ResilienceService.retentionPercent), PeerManager.getPeers().size()); } - - private void clearPeers() { - for (PeerConnection p : PeerManager.getPeers()) { - PeerManager.remove(p.getChannel()); - } - } - - @After - public void destroy() { - Args.clearParam(); - context.destroy(); - } } \ No newline at end of file diff --git a/framework/src/test/java/org/tron/core/net/services/SyncServiceTest.java b/framework/src/test/java/org/tron/core/net/services/SyncServiceTest.java index c2883fb349d..785262e3210 100644 --- a/framework/src/test/java/org/tron/core/net/services/SyncServiceTest.java +++ b/framework/src/test/java/org/tron/core/net/services/SyncServiceTest.java @@ -19,6 +19,7 @@ import org.springframework.context.ApplicationContext; import org.tron.common.application.TronApplicationContext; import org.tron.common.utils.ReflectUtils; +import org.tron.common.utils.Sha256Hash; import org.tron.core.Constant; import org.tron.core.capsule.BlockCapsule; import org.tron.core.config.DefaultConfig; @@ -64,6 +65,9 @@ public void init() throws Exception { */ @After public void destroy() { + for (PeerConnection p : PeerManager.getPeers()) { + PeerManager.remove(p.getChannel()); + } Args.clearParam(); context.destroy(); } @@ -91,6 +95,13 @@ public void testStartSync() { ReflectUtils.setFieldValue(peer, "tronState", TronState.INIT); + try { + peer.setBlockBothHave(new BlockCapsule.BlockId(Sha256Hash.ZERO_HASH, -1)); + service.syncNext(peer); + } catch (Exception e) { + // no need to deal with + } + service.startSync(peer); } catch (Exception e) { // no need to deal with diff --git a/framework/src/test/java/org/tron/core/services/DelegationServiceTest.java b/framework/src/test/java/org/tron/core/services/DelegationServiceTest.java index 5c898eb42d6..0ce4ba4d67d 100644 --- a/framework/src/test/java/org/tron/core/services/DelegationServiceTest.java +++ b/framework/src/test/java/org/tron/core/services/DelegationServiceTest.java @@ -4,49 +4,28 @@ import static org.tron.common.utils.client.Parameter.CommonConstant.ADD_PRE_FIX_BYTE_MAINNET; import com.google.protobuf.ByteString; -import io.grpc.ManagedChannelBuilder; +import javax.annotation.Resource; import lombok.extern.slf4j.Slf4j; import org.junit.Assert; -import org.tron.api.GrpcAPI.BytesMessage; -import org.tron.api.GrpcAPI.TransactionExtention; -import org.tron.api.WalletGrpc; -import org.tron.api.WalletGrpc.WalletBlockingStub; -import org.tron.common.application.TronApplicationContext; +import org.junit.BeforeClass; +import org.junit.Test; +import org.tron.common.BaseTest; +import org.tron.core.Constant; import org.tron.core.Wallet; import org.tron.core.capsule.AccountCapsule; -import org.tron.core.db.Manager; +import org.tron.core.config.args.Args; import org.tron.core.service.MortgageService; -import org.tron.protos.contract.StorageContract.UpdateBrokerageContract; @Slf4j -public class DelegationServiceTest { +public class DelegationServiceTest extends BaseTest { - private static String fullnode = "127.0.0.1:50051"; - private MortgageService mortgageService; - private Manager manager; + @Resource + protected MortgageService mortgageService; - public DelegationServiceTest(TronApplicationContext context) { - mortgageService = context.getBean(MortgageService.class); - manager = context.getBean(Manager.class); - } - - public static void testGrpc() { - WalletBlockingStub walletStub = WalletGrpc - .newBlockingStub(ManagedChannelBuilder.forTarget(fullnode) - .usePlaintext() - .build()); - BytesMessage.Builder builder = BytesMessage.newBuilder(); - builder.setValue(ByteString.copyFromUtf8("TLTDZBcPoJ8tZ6TTEeEqEvwYFk2wgotSfD")); - System.out - .println("getBrokerageInfo: " + walletStub.getBrokerageInfo(builder.build()).getNum()); - System.out.println("getRewardInfo: " + walletStub.getRewardInfo(builder.build()).getNum()); - UpdateBrokerageContract.Builder updateBrokerageContract = UpdateBrokerageContract.newBuilder(); - updateBrokerageContract.setOwnerAddress( - ByteString.copyFrom(decodeFromBase58Check("TN3zfjYUmMFK3ZsHSsrdJoNRtGkQmZLBLz"))) - .setBrokerage(10); - TransactionExtention transactionExtention = walletStub - .updateBrokerage(updateBrokerageContract.build()); - System.out.println("UpdateBrokerage: " + transactionExtention); + @BeforeClass + public static void init() { + Args.setParam(new String[] {"--output-directory", dbPath(), "--debug"}, + Constant.TESTNET_CONF); } private void testPay(int cycle) { @@ -59,7 +38,7 @@ private void testPay(int cycle) { mortgageService.payStandbyWitness(); Wallet.setAddressPreFixByte(ADD_PRE_FIX_BYTE_MAINNET); byte[] sr1 = decodeFromBase58Check("TLTDZBcPoJ8tZ6TTEeEqEvwYFk2wgotSfD"); - long value = manager.getDelegationStore().getReward(cycle, sr1); + long value = dbManager.getDelegationStore().getReward(cycle, sr1); long tmp = 0; for (int i = 0; i < 27; i++) { tmp += 100000000 + i; @@ -73,47 +52,47 @@ private void testPay(int cycle) { expect += 32000000; brokerageAmount = (long) (rate * 32000000); expect -= brokerageAmount; - value = manager.getDelegationStore().getReward(cycle, sr1); + value = dbManager.getDelegationStore().getReward(cycle, sr1); Assert.assertEquals(expect, value); } private void testWithdraw() { //init - manager.getDynamicPropertiesStore().saveCurrentCycleNumber(1); + dbManager.getDynamicPropertiesStore().saveCurrentCycleNumber(1); testPay(1); - manager.getDynamicPropertiesStore().saveCurrentCycleNumber(2); + dbManager.getDynamicPropertiesStore().saveCurrentCycleNumber(2); testPay(2); byte[] sr1 = decodeFromBase58Check("THKJYuUmMKKARNf7s2VT51g5uPY6KEqnat"); - AccountCapsule accountCapsule = manager.getAccountStore().get(sr1); + AccountCapsule accountCapsule = dbManager.getAccountStore().get(sr1); byte[] sr27 = decodeFromBase58Check("TLTDZBcPoJ8tZ6TTEeEqEvwYFk2wgotSfD"); accountCapsule.addVotes(ByteString.copyFrom(sr27), 10000000); - manager.getAccountStore().put(sr1, accountCapsule); + dbManager.getAccountStore().put(sr1, accountCapsule); // long allowance = accountCapsule.getAllowance(); long value = mortgageService.queryReward(sr1) - allowance; - long reward1 = (long) ((double) manager.getDelegationStore().getReward(0, sr27) / 100000000 + long reward1 = (long) ((double) dbManager.getDelegationStore().getReward(0, sr27) / 100000000 * 10000000); - long reward2 = (long) ((double) manager.getDelegationStore().getReward(1, sr27) / 100000000 + long reward2 = (long) ((double) dbManager.getDelegationStore().getReward(1, sr27) / 100000000 * 10000000); long reward = reward1 + reward2; - System.out.println("testWithdraw:" + value + ", reward:" + reward); Assert.assertEquals(reward, value); mortgageService.withdrawReward(sr1); - accountCapsule = manager.getAccountStore().get(sr1); + accountCapsule = dbManager.getAccountStore().get(sr1); allowance = accountCapsule.getAllowance() - allowance; System.out.println("withdrawReward:" + allowance); Assert.assertEquals(reward, allowance); } + @Test public void test() { - manager.getDynamicPropertiesStore().saveChangeDelegation(1); - manager.getDynamicPropertiesStore().saveConsensusLogicOptimization(1); + dbManager.getDynamicPropertiesStore().saveChangeDelegation(1); + dbManager.getDynamicPropertiesStore().saveConsensusLogicOptimization(1); byte[] sr27 = decodeFromBase58Check("TLTDZBcPoJ8tZ6TTEeEqEvwYFk2wgotSfD"); - manager.getDelegationStore().setBrokerage(0, sr27, 10); - manager.getDelegationStore().setBrokerage(1, sr27, 20); - manager.getDelegationStore().setWitnessVote(0, sr27, 100000000); - manager.getDelegationStore().setWitnessVote(1, sr27, 100000000); - manager.getDelegationStore().setWitnessVote(2, sr27, 100000000); + dbManager.getDelegationStore().setBrokerage(0, sr27, 10); + dbManager.getDelegationStore().setBrokerage(1, sr27, 20); + dbManager.getDelegationStore().setWitnessVote(0, sr27, 100000000); + dbManager.getDelegationStore().setWitnessVote(1, sr27, 100000000); + dbManager.getDelegationStore().setWitnessVote(2, sr27, 100000000); testPay(0); testWithdraw(); } diff --git a/framework/src/test/java/org/tron/core/services/NodeInfoServiceTest.java b/framework/src/test/java/org/tron/core/services/NodeInfoServiceTest.java index be0a96f632b..2ea410f93f1 100644 --- a/framework/src/test/java/org/tron/core/services/NodeInfoServiceTest.java +++ b/framework/src/test/java/org/tron/core/services/NodeInfoServiceTest.java @@ -1,35 +1,56 @@ package org.tron.core.services; -import static org.mockito.Mockito.mock; - import com.alibaba.fastjson.JSON; import com.google.protobuf.ByteString; import java.net.InetSocketAddress; +import javax.annotation.Resource; import lombok.extern.slf4j.Slf4j; +import org.junit.After; import org.junit.Assert; -import org.mockito.Mockito; -import org.tron.common.application.TronApplicationContext; +import org.junit.BeforeClass; +import org.junit.Test; +import org.springframework.context.ApplicationContext; +import org.tron.common.BaseTest; import org.tron.common.entity.NodeInfo; +import org.tron.common.utils.PublicMethod; +import org.tron.common.utils.ReflectUtils; import org.tron.common.utils.Sha256Hash; +import org.tron.core.Constant; import org.tron.core.capsule.BlockCapsule; +import org.tron.core.config.args.Args; import org.tron.core.net.P2pEventHandlerImpl; +import org.tron.core.net.TronNetService; +import org.tron.core.net.peer.PeerManager; +import org.tron.p2p.P2pConfig; import org.tron.p2p.connection.Channel; import org.tron.program.Version; @Slf4j -public class NodeInfoServiceTest { +public class NodeInfoServiceTest extends BaseTest { - private NodeInfoService nodeInfoService; - private WitnessProductBlockService witnessProductBlockService; + @Resource + protected NodeInfoService nodeInfoService; + @Resource + protected WitnessProductBlockService witnessProductBlockService; + @Resource private P2pEventHandlerImpl p2pEventHandler; + @Resource + private TronNetService tronNetService; + + + @BeforeClass + public static void init() { + Args.setParam(new String[] {"--output-directory", dbPath(), "--debug"}, + Constant.TEST_CONF); + } - public NodeInfoServiceTest(TronApplicationContext context) { - nodeInfoService = context.getBean("nodeInfoService", NodeInfoService.class); - witnessProductBlockService = context.getBean(WitnessProductBlockService.class); - p2pEventHandler = context.getBean(P2pEventHandlerImpl.class); + @After + public void clearPeers() { + closePeer(); } + @Test public void test() { BlockCapsule blockCapsule1 = new BlockCapsule(1, Sha256Hash.ZERO_HASH, 100, ByteString.EMPTY); @@ -38,18 +59,31 @@ public void test() { witnessProductBlockService.validWitnessProductTwoBlock(blockCapsule1); witnessProductBlockService.validWitnessProductTwoBlock(blockCapsule2); - //add peer - InetSocketAddress a1 = new InetSocketAddress("127.0.0.1", 10001); - Channel c1 = mock(Channel.class); - Mockito.when(c1.getInetSocketAddress()).thenReturn(a1); - Mockito.when(c1.getInetAddress()).thenReturn(a1.getAddress()); - p2pEventHandler.onConnect(c1); + addPeer(); //test setConnectInfo NodeInfo nodeInfo = nodeInfoService.getNodeInfo(); Assert.assertEquals(nodeInfo.getConfigNodeInfo().getCodeVersion(), Version.getVersion()); - Assert.assertEquals(nodeInfo.getCheatWitnessInfoMap().size(), 1); + Assert.assertEquals(1, nodeInfo.getCheatWitnessInfoMap().size()); logger.info("{}", JSON.toJSONString(nodeInfo)); } + private void addPeer() { + int port = PublicMethod.chooseRandomPort(); + P2pConfig p2pConfig = new P2pConfig(); + p2pConfig.setIp("127.0.0.1"); + p2pConfig.setPort(port); + ReflectUtils.setFieldValue(tronNetService, "p2pConfig", p2pConfig); + TronNetService.getP2pService().start(p2pConfig); + + ApplicationContext ctx = (ApplicationContext) ReflectUtils.getFieldObject(p2pEventHandler, + "ctx"); + InetSocketAddress inetSocketAddress1 = + new InetSocketAddress("127.0.0.1", 10001); + Channel c1 = new Channel(); + ReflectUtils.setFieldValue(c1, "inetSocketAddress", inetSocketAddress1); + ReflectUtils.setFieldValue(c1, "inetAddress", inetSocketAddress1.getAddress()); + + PeerManager.add(ctx, c1); + } } diff --git a/framework/src/test/java/org/tron/core/services/ProposalServiceTest.java b/framework/src/test/java/org/tron/core/services/ProposalServiceTest.java index 300a38a0916..e81c75948b7 100644 --- a/framework/src/test/java/org/tron/core/services/ProposalServiceTest.java +++ b/framework/src/test/java/org/tron/core/services/ProposalServiceTest.java @@ -1,7 +1,9 @@ package org.tron.core.services; +import static org.tron.core.Constant.MAX_PROPOSAL_EXPIRE_TIME; import static org.tron.core.utils.ProposalUtil.ProposalType.CONSENSUS_LOGIC_OPTIMIZATION; import static org.tron.core.utils.ProposalUtil.ProposalType.ENERGY_FEE; +import static org.tron.core.utils.ProposalUtil.ProposalType.PROPOSAL_EXPIRE_TIME; import static org.tron.core.utils.ProposalUtil.ProposalType.TRANSACTION_FEE; import static org.tron.core.utils.ProposalUtil.ProposalType.WITNESS_127_PAY_PER_BLOCK; @@ -13,6 +15,7 @@ import org.junit.BeforeClass; import org.junit.Test; import org.tron.common.BaseTest; +import org.tron.common.parameter.CommonParameter; import org.tron.core.Constant; import org.tron.core.capsule.ProposalCapsule; import org.tron.core.config.args.Args; @@ -131,4 +134,21 @@ public void testUpdateConsensusLogicOptimization() { Assert.assertTrue(dbManager.getDynamicPropertiesStore().disableJavaLangMath()); } + @Test + public void testProposalExpireTime() { + long defaultWindow = dbManager.getDynamicPropertiesStore().getProposalExpireTime(); + long proposalExpireTime = CommonParameter.getInstance().getProposalExpireTime(); + Assert.assertEquals(proposalExpireTime, defaultWindow); + + Proposal proposal = Proposal.newBuilder().putParameters(PROPOSAL_EXPIRE_TIME.getCode(), + 31536000000L).build(); + ProposalCapsule proposalCapsule = new ProposalCapsule(proposal); + proposalCapsule.setExpirationTime(1627279200000L); + boolean result = ProposalService.process(dbManager, proposalCapsule); + Assert.assertTrue(result); + + long window = dbManager.getDynamicPropertiesStore().getProposalExpireTime(); + Assert.assertEquals(MAX_PROPOSAL_EXPIRE_TIME - 3000, window); + } + } \ No newline at end of file diff --git a/framework/src/test/java/org/tron/core/services/RpcApiServicesTest.java b/framework/src/test/java/org/tron/core/services/RpcApiServicesTest.java index 3ae090d3caf..c873613bb91 100644 --- a/framework/src/test/java/org/tron/core/services/RpcApiServicesTest.java +++ b/framework/src/test/java/org/tron/core/services/RpcApiServicesTest.java @@ -11,12 +11,17 @@ import io.grpc.ManagedChannelBuilder; import java.io.IOException; import java.util.Objects; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.TimeUnit; import org.junit.AfterClass; +import org.junit.Assert; import org.junit.BeforeClass; import org.junit.ClassRule; import org.junit.FixMethodOrder; +import org.junit.Rule; import org.junit.Test; import org.junit.rules.TemporaryFolder; +import org.junit.rules.Timeout; import org.junit.runners.MethodSorters; import org.tron.api.DatabaseGrpc; import org.tron.api.DatabaseGrpc.DatabaseBlockingStub; @@ -49,9 +54,11 @@ import org.tron.common.application.Application; import org.tron.common.application.ApplicationFactory; import org.tron.common.application.TronApplicationContext; +import org.tron.common.es.ExecutorServiceManager; import org.tron.common.utils.ByteArray; import org.tron.common.utils.PublicMethod; import org.tron.common.utils.Sha256Hash; +import org.tron.common.utils.TimeoutInterceptor; import org.tron.core.Constant; import org.tron.core.Wallet; import org.tron.core.capsule.AccountCapsule; @@ -60,8 +67,6 @@ import org.tron.core.config.DefaultConfig; import org.tron.core.config.args.Args; import org.tron.core.db.Manager; -import org.tron.core.services.interfaceOnPBFT.RpcApiServiceOnPBFT; -import org.tron.core.services.interfaceOnSolidity.RpcApiServiceOnSolidity; import org.tron.protos.Protocol; import org.tron.protos.Protocol.Account; import org.tron.protos.Protocol.Block; @@ -111,7 +116,12 @@ @FixMethodOrder(MethodSorters.NAME_ASCENDING) public class RpcApiServicesTest { + + private static Application appTest; private static TronApplicationContext context; + private static ManagedChannel channelFull = null; + private static ManagedChannel channelPBFT = null; + private static ManagedChannel channelSolidity = null; private static DatabaseBlockingStub databaseBlockingStubFull = null; private static DatabaseBlockingStub databaseBlockingStubSolidity = null; private static DatabaseBlockingStub databaseBlockingStubPBFT = null; @@ -120,6 +130,8 @@ public class RpcApiServicesTest { private static WalletSolidityBlockingStub blockingStubPBFT = null; @ClassRule public static TemporaryFolder temporaryFolder = new TemporaryFolder(); + @Rule + public Timeout timeout = new Timeout(30, TimeUnit.SECONDS); private static ByteString ownerAddress; private static ByteString sk; private static ByteString ask; @@ -130,9 +142,14 @@ public class RpcApiServicesTest { private static ByteString ivk; private static ByteString d; + private static ExecutorService executorService; + private static final String executorName = "rpc-test-executor"; + @BeforeClass public static void init() throws IOException { - Args.setParam(new String[]{"-d", temporaryFolder.newFolder().toString()}, Constant.TEST_CONF); + Args.setParam(new String[] {"-d", temporaryFolder.newFolder().toString()}, Constant.TEST_CONF); + Assert.assertEquals(5, getInstance().getRpcMaxRstStream()); + Assert.assertEquals(10, getInstance().getRpcSecondsPerWindow()); String OWNER_ADDRESS = Wallet.getAddressPreFixString() + "548794500882809695a8a687866e76d4271a1abc"; getInstance().setRpcEnable(true); @@ -144,21 +161,30 @@ public static void init() throws IOException { getInstance().setMetricsPrometheusPort(PublicMethod.chooseRandomPort()); getInstance().setMetricsPrometheusEnable(true); getInstance().setP2pDisable(true); - String fullNode = String.format("%s:%d", getInstance().getNodeLanIp(), + String fullNode = String.format("%s:%d", Constant.LOCAL_HOST, getInstance().getRpcPort()); - String solidityNode = String.format("%s:%d", getInstance().getNodeLanIp(), + String solidityNode = String.format("%s:%d", Constant.LOCAL_HOST, getInstance().getRpcOnSolidityPort()); - String pBFTNode = String.format("%s:%d", getInstance().getNodeLanIp(), + String pBFTNode = String.format("%s:%d", Constant.LOCAL_HOST, getInstance().getRpcOnPBFTPort()); - ManagedChannel channelFull = ManagedChannelBuilder.forTarget(fullNode) + executorService = ExecutorServiceManager.newFixedThreadPool( + executorName, 3); + + channelFull = ManagedChannelBuilder.forTarget(fullNode) .usePlaintext() + .executor(executorService) + .intercept(new TimeoutInterceptor(5000)) .build(); - ManagedChannel channelPBFT = ManagedChannelBuilder.forTarget(pBFTNode) + channelPBFT = ManagedChannelBuilder.forTarget(pBFTNode) .usePlaintext() + .executor(executorService) + .intercept(new TimeoutInterceptor(5000)) .build(); - ManagedChannel channelSolidity = ManagedChannelBuilder.forTarget(solidityNode) + channelSolidity = ManagedChannelBuilder.forTarget(solidityNode) .usePlaintext() + .executor(executorService) + .intercept(new TimeoutInterceptor(5000)) .build(); context = new TronApplicationContext(DefaultConfig.class); databaseBlockingStubFull = DatabaseGrpc.newBlockingStub(channelFull); @@ -176,16 +202,41 @@ public static void init() throws IOException { manager.getAccountStore().put(ownerCapsule.createDbKey(), ownerCapsule); manager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); manager.getDynamicPropertiesStore().saveAllowShieldedTRC20Transaction(1); - Application appTest = ApplicationFactory.create(context); + appTest = ApplicationFactory.create(context); appTest.startup(); } @AfterClass public static void destroy() { + shutdownChannel(channelFull); + shutdownChannel(channelPBFT); + shutdownChannel(channelSolidity); + + if (executorService != null) { + ExecutorServiceManager.shutdownAndAwaitTermination( + executorService, executorName); + executorService = null; + } + context.close(); Args.clearParam(); } + private static void shutdownChannel(ManagedChannel channel) { + if (channel == null) { + return; + } + try { + channel.shutdown(); + if (!channel.awaitTermination(5, TimeUnit.SECONDS)) { + channel.shutdownNow(); + } + } catch (InterruptedException e) { + channel.shutdownNow(); + Thread.currentThread().interrupt(); + } + } + @Test public void testGetBlockByNum() { NumberMessage message = NumberMessage.newBuilder().setNum(0).build(); @@ -233,6 +284,14 @@ public void testListWitnesses() { assertNotNull(blockingStubPBFT.listWitnesses(message)); } + @Test + public void testGetPaginatedNowWitnessList() { + PaginatedMessage paginatedMessage = PaginatedMessage.newBuilder() + .setOffset(0).setLimit(5).build(); + assertNotNull(blockingStubFull.getPaginatedNowWitnessList(paginatedMessage)); + assertNotNull(blockingStubSolidity.getPaginatedNowWitnessList(paginatedMessage)); + } + @Test public void testGetAssetIssueList() { EmptyMessage message = EmptyMessage.newBuilder().build(); diff --git a/framework/src/test/java/org/tron/core/services/WalletApiTest.java b/framework/src/test/java/org/tron/core/services/WalletApiTest.java index 8890d4bfd9e..f9c95f018c4 100644 --- a/framework/src/test/java/org/tron/core/services/WalletApiTest.java +++ b/framework/src/test/java/org/tron/core/services/WalletApiTest.java @@ -2,19 +2,23 @@ import io.grpc.ManagedChannelBuilder; import java.io.IOException; +import java.util.concurrent.TimeUnit; import lombok.extern.slf4j.Slf4j; -import org.junit.After; +import org.junit.AfterClass; import org.junit.Assert; import org.junit.BeforeClass; import org.junit.ClassRule; +import org.junit.Rule; import org.junit.Test; import org.junit.rules.TemporaryFolder; +import org.junit.rules.Timeout; import org.tron.api.GrpcAPI.EmptyMessage; import org.tron.api.WalletGrpc; import org.tron.common.application.Application; import org.tron.common.application.ApplicationFactory; import org.tron.common.application.TronApplicationContext; import org.tron.common.utils.PublicMethod; +import org.tron.common.utils.TimeoutInterceptor; import org.tron.core.Constant; import org.tron.core.config.DefaultConfig; import org.tron.core.config.args.Args; @@ -26,13 +30,16 @@ public class WalletApiTest { @ClassRule public static TemporaryFolder temporaryFolder = new TemporaryFolder(); + @Rule + public Timeout timeout = new Timeout(30, TimeUnit.SECONDS); + private static TronApplicationContext context; private static Application appT; @BeforeClass public static void init() throws IOException { - Args.setParam(new String[]{ "-d", temporaryFolder.newFolder().toString(), + Args.setParam(new String[] {"-d", temporaryFolder.newFolder().toString(), "--p2p-disable", "true"}, Constant.TEST_CONF); Args.getInstance().setRpcPort(PublicMethod.chooseRandomPort()); Args.getInstance().setRpcEnable(true); @@ -45,18 +52,32 @@ public static void init() throws IOException { public void listNodesTest() { String fullNode = String.format("%s:%d", "127.0.0.1", Args.getInstance().getRpcPort()); - WalletGrpc.WalletBlockingStub walletStub = WalletGrpc - .newBlockingStub(ManagedChannelBuilder.forTarget(fullNode) - .usePlaintext() - .build()); - Assert.assertTrue(walletStub.listNodes(EmptyMessage.getDefaultInstance()) - .getNodesList().isEmpty()); + io.grpc.ManagedChannel channel = ManagedChannelBuilder.forTarget(fullNode) + .usePlaintext() + .intercept(new TimeoutInterceptor(5000)) + .build(); + try { + WalletGrpc.WalletBlockingStub walletStub = WalletGrpc.newBlockingStub(channel); + Assert.assertTrue(walletStub.listNodes(EmptyMessage.getDefaultInstance()) + .getNodesList().isEmpty()); + } finally { + // Properly shutdown the gRPC channel to prevent resource leaks + channel.shutdown(); + try { + if (!channel.awaitTermination(5, java.util.concurrent.TimeUnit.SECONDS)) { + channel.shutdownNow(); + } + } catch (InterruptedException e) { + channel.shutdownNow(); + Thread.currentThread().interrupt(); + } + } } - @After - public void destroy() { - Args.clearParam(); + @AfterClass + public static void destroy() { context.destroy(); + Args.clearParam(); } } diff --git a/framework/src/test/java/org/tron/core/services/filter/HttpApiAccessFilterTest.java b/framework/src/test/java/org/tron/core/services/filter/HttpApiAccessFilterTest.java index 420d890aa48..0d2c9c9ae78 100644 --- a/framework/src/test/java/org/tron/core/services/filter/HttpApiAccessFilterTest.java +++ b/framework/src/test/java/org/tron/core/services/filter/HttpApiAccessFilterTest.java @@ -38,7 +38,7 @@ public class HttpApiAccessFilterTest extends BaseTest { static { Args.setParam(new String[]{"-d", dbPath()}, Constant.TEST_CONF); - Args.getInstance().setFullNodeAllowShieldedTransactionArgs(false); + Args.getInstance().setAllowShieldedTransactionApi(false); Args.getInstance().setFullNodeHttpEnable(true); Args.getInstance().setFullNodeHttpPort(PublicMethod.chooseRandomPort()); Args.getInstance().setPBFTHttpEnable(true); diff --git a/framework/src/test/java/org/tron/core/services/filter/LiteFnQueryGrpcInterceptorTest.java b/framework/src/test/java/org/tron/core/services/filter/LiteFnQueryGrpcInterceptorTest.java index 84869ea0750..b3cd5844b8d 100644 --- a/framework/src/test/java/org/tron/core/services/filter/LiteFnQueryGrpcInterceptorTest.java +++ b/framework/src/test/java/org/tron/core/services/filter/LiteFnQueryGrpcInterceptorTest.java @@ -14,6 +14,7 @@ import org.junit.Test; import org.junit.rules.ExpectedException; import org.junit.rules.TemporaryFolder; +import org.junit.rules.Timeout; import org.tron.api.GrpcAPI; import org.tron.api.WalletGrpc; import org.tron.api.WalletSolidityGrpc; @@ -21,13 +22,11 @@ import org.tron.common.application.ApplicationFactory; import org.tron.common.application.TronApplicationContext; import org.tron.common.utils.PublicMethod; +import org.tron.common.utils.TimeoutInterceptor; import org.tron.core.ChainBaseManager; import org.tron.core.Constant; import org.tron.core.config.DefaultConfig; import org.tron.core.config.args.Args; -import org.tron.core.services.RpcApiService; -import org.tron.core.services.interfaceOnPBFT.RpcApiServiceOnPBFT; -import org.tron.core.services.interfaceOnSolidity.RpcApiServiceOnSolidity; @Slf4j public class LiteFnQueryGrpcInterceptorTest { @@ -49,12 +48,15 @@ public class LiteFnQueryGrpcInterceptorTest { @ClassRule public static final TemporaryFolder temporaryFolder = new TemporaryFolder(); + @Rule + public Timeout timeout = new Timeout(30, TimeUnit.SECONDS); + /** * init logic. */ @BeforeClass public static void init() throws IOException { - Args.setParam(new String[]{"-d", temporaryFolder.newFolder().toString()}, Constant.TEST_CONF); + Args.setParam(new String[] {"-d", temporaryFolder.newFolder().toString()}, Constant.TEST_CONF); Args.getInstance().setRpcEnable(true); Args.getInstance().setRpcPort(PublicMethod.chooseRandomPort()); Args.getInstance().setRpcSolidityEnable(true); @@ -62,23 +64,25 @@ public static void init() throws IOException { Args.getInstance().setRpcPBFTEnable(true); Args.getInstance().setRpcOnPBFTPort(PublicMethod.chooseRandomPort()); Args.getInstance().setP2pDisable(true); - String fullnode = String.format("%s:%d", Args.getInstance().getNodeLanIp(), - Args.getInstance().getRpcPort()); - String solidityNode = String.format("%s:%d", Args.getInstance().getNodeLanIp(), - Args.getInstance().getRpcOnSolidityPort()); - String pBFTNode = String.format("%s:%d", Args.getInstance().getNodeLanIp(), + String fullnode = String.format("%s:%d", Constant.LOCAL_HOST, + Args.getInstance().getRpcPort()); + String solidityNode = String.format("%s:%d", Constant.LOCAL_HOST, + Args.getInstance().getRpcOnSolidityPort()); + String pBFTNode = String.format("%s:%d", Constant.LOCAL_HOST, Args.getInstance().getRpcOnPBFTPort()); channelFull = ManagedChannelBuilder.forTarget(fullnode) - .usePlaintext() - .build(); + .usePlaintext() + .intercept(new TimeoutInterceptor(5000)) + .build(); channelSolidity = ManagedChannelBuilder.forTarget(solidityNode) .usePlaintext() + .intercept(new TimeoutInterceptor(5000)) .build(); channelpBFT = ManagedChannelBuilder.forTarget(pBFTNode) - .usePlaintext() - .build(); + .usePlaintext() + .intercept(new TimeoutInterceptor(5000)) + .build(); context = new TronApplicationContext(DefaultConfig.class); - context.registerShutdownHook(); blockingStubFull = WalletGrpc.newBlockingStub(channelFull); blockingStubSolidity = WalletSolidityGrpc.newBlockingStub(channelSolidity); blockingStubpBFT = WalletSolidityGrpc.newBlockingStub(channelpBFT); @@ -93,13 +97,13 @@ public static void init() throws IOException { @AfterClass public static void destroy() throws InterruptedException { if (channelFull != null) { - channelFull.shutdown().awaitTermination(5, TimeUnit.SECONDS); + channelFull.shutdownNow(); } if (channelSolidity != null) { - channelSolidity.shutdown().awaitTermination(5, TimeUnit.SECONDS); + channelSolidity.shutdownNow(); } if (channelpBFT != null) { - channelpBFT.shutdown().awaitTermination(5, TimeUnit.SECONDS); + channelpBFT.shutdownNow(); } context.close(); Args.clearParam(); diff --git a/framework/src/test/java/org/tron/core/services/filter/LiteFnQueryHttpFilterTest.java b/framework/src/test/java/org/tron/core/services/filter/LiteFnQueryHttpFilterTest.java index 0f0bdf1eb1f..978042a8578 100644 --- a/framework/src/test/java/org/tron/core/services/filter/LiteFnQueryHttpFilterTest.java +++ b/framework/src/test/java/org/tron/core/services/filter/LiteFnQueryHttpFilterTest.java @@ -31,7 +31,7 @@ public class LiteFnQueryHttpFilterTest extends BaseTest { static { Args.setParam(new String[]{"-d", dbPath()}, Constant.TEST_CONF); - Args.getInstance().setFullNodeAllowShieldedTransactionArgs(false); + Args.getInstance().setAllowShieldedTransactionApi(false); Args.getInstance().setRpcEnable(false); Args.getInstance().setRpcSolidityEnable(false); Args.getInstance().setRpcPBFTEnable(false); diff --git a/framework/src/test/java/org/tron/core/services/filter/RpcApiAccessInterceptorTest.java b/framework/src/test/java/org/tron/core/services/filter/RpcApiAccessInterceptorTest.java index 900ca304e7d..2e02125e014 100644 --- a/framework/src/test/java/org/tron/core/services/filter/RpcApiAccessInterceptorTest.java +++ b/framework/src/test/java/org/tron/core/services/filter/RpcApiAccessInterceptorTest.java @@ -14,12 +14,15 @@ import java.util.Collections; import java.util.List; import java.util.Objects; +import java.util.concurrent.TimeUnit; import lombok.extern.slf4j.Slf4j; import org.junit.AfterClass; import org.junit.BeforeClass; import org.junit.ClassRule; +import org.junit.Rule; import org.junit.Test; import org.junit.rules.TemporaryFolder; +import org.junit.rules.Timeout; import org.tron.api.GrpcAPI.BlockExtention; import org.tron.api.GrpcAPI.BlockReq; import org.tron.api.GrpcAPI.BytesMessage; @@ -32,24 +35,29 @@ import org.tron.common.application.ApplicationFactory; import org.tron.common.application.TronApplicationContext; import org.tron.common.utils.PublicMethod; +import org.tron.common.utils.TimeoutInterceptor; import org.tron.core.Constant; import org.tron.core.config.DefaultConfig; import org.tron.core.config.args.Args; import org.tron.core.services.RpcApiService; -import org.tron.core.services.interfaceOnPBFT.RpcApiServiceOnPBFT; -import org.tron.core.services.interfaceOnSolidity.RpcApiServiceOnSolidity; import org.tron.protos.Protocol.Transaction; @Slf4j public class RpcApiAccessInterceptorTest { private static TronApplicationContext context; + private static ManagedChannel channelFull = null; + private static ManagedChannel channelPBFT = null; + private static ManagedChannel channelSolidity = null; private static WalletGrpc.WalletBlockingStub blockingStubFull = null; private static WalletSolidityGrpc.WalletSolidityBlockingStub blockingStubSolidity = null; private static WalletSolidityGrpc.WalletSolidityBlockingStub blockingStubPBFT = null; @ClassRule public static final TemporaryFolder temporaryFolder = new TemporaryFolder(); + @Rule + public Timeout timeout = new Timeout(30, TimeUnit.SECONDS); + /** * init logic. */ @@ -63,21 +71,24 @@ public static void init() throws IOException { Args.getInstance().setRpcPBFTEnable(true); Args.getInstance().setRpcOnPBFTPort(PublicMethod.chooseRandomPort()); Args.getInstance().setP2pDisable(true); - String fullNode = String.format("%s:%d", Args.getInstance().getNodeLanIp(), + String fullNode = String.format("%s:%d", Constant.LOCAL_HOST, Args.getInstance().getRpcPort()); - String solidityNode = String.format("%s:%d", Args.getInstance().getNodeLanIp(), + String solidityNode = String.format("%s:%d", Constant.LOCAL_HOST, Args.getInstance().getRpcOnSolidityPort()); - String pBFTNode = String.format("%s:%d", Args.getInstance().getNodeLanIp(), + String pBFTNode = String.format("%s:%d", Constant.LOCAL_HOST, Args.getInstance().getRpcOnPBFTPort()); - ManagedChannel channelFull = ManagedChannelBuilder.forTarget(fullNode) + channelFull = ManagedChannelBuilder.forTarget(fullNode) .usePlaintext() + .intercept(new TimeoutInterceptor(5000)) .build(); - ManagedChannel channelPBFT = ManagedChannelBuilder.forTarget(pBFTNode) + channelPBFT = ManagedChannelBuilder.forTarget(pBFTNode) .usePlaintext() + .intercept(new TimeoutInterceptor(5000)) .build(); - ManagedChannel channelSolidity = ManagedChannelBuilder.forTarget(solidityNode) + channelSolidity = ManagedChannelBuilder.forTarget(solidityNode) .usePlaintext() + .intercept(new TimeoutInterceptor(5000)) .build(); context = new TronApplicationContext(DefaultConfig.class); @@ -95,6 +106,15 @@ public static void init() throws IOException { */ @AfterClass public static void destroy() { + if (channelFull != null) { + channelFull.shutdownNow(); + } + if (channelPBFT != null) { + channelPBFT.shutdownNow(); + } + if (channelSolidity != null) { + channelSolidity.shutdownNow(); + } context.close(); Args.clearParam(); } diff --git a/framework/src/test/java/org/tron/core/services/http/GetRewardServletTest.java b/framework/src/test/java/org/tron/core/services/http/GetRewardServletTest.java index 404e154a4c3..bd367fc3700 100644 --- a/framework/src/test/java/org/tron/core/services/http/GetRewardServletTest.java +++ b/framework/src/test/java/org/tron/core/services/http/GetRewardServletTest.java @@ -58,7 +58,7 @@ public MockHttpServletRequest createRequest(String contentType) { @Before public void init() { manager.getDynamicPropertiesStore().saveChangeDelegation(1); - byte[] sr = decodeFromBase58Check("27VZHn9PFZwNh7o2EporxmLkpe157iWZVkh"); + byte[] sr = decodeFromBase58Check("27bi7CD8d94AgXY3XFS9A9vx78Si5MqrECz"); delegationStore.setBrokerage(0, sr, 10); delegationStore.setWitnessVote(0, sr, 100000000); } @@ -66,7 +66,7 @@ public void init() { @Test public void getRewardValueByJsonTest() { int expect = 138181; - String jsonParam = "{\"address\": \"27VZHn9PFZwNh7o2EporxmLkpe157iWZVkh\"}"; + String jsonParam = "{\"address\": \"27bi7CD8d94AgXY3XFS9A9vx78Si5MqrECz\"}"; MockHttpServletRequest request = createRequest("application/json"); MockHttpServletResponse response = new MockHttpServletResponse(); request.setContent(jsonParam.getBytes()); @@ -84,7 +84,7 @@ public void getRewardValueByJsonTest() { @Test public void getRewardByJsonUTF8Test() { int expect = 138181; - String jsonParam = "{\"address\": \"27VZHn9PFZwNh7o2EporxmLkpe157iWZVkh\"}"; + String jsonParam = "{\"address\": \"27bi7CD8d94AgXY3XFS9A9vx78Si5MqrECz\"}"; MockHttpServletRequest request = createRequest("application/json; charset=utf-8"); MockHttpServletResponse response = new MockHttpServletResponse(); request.setContent(jsonParam.getBytes()); @@ -105,7 +105,7 @@ public void getRewardValueTest() { MockHttpServletRequest request = createRequest("application/x-www-form-urlencoded"); MockHttpServletResponse response = new MockHttpServletResponse(); mortgageService.payStandbyWitness(); - request.addParameter("address", "27VZHn9PFZwNh7o2EporxmLkpe157iWZVkh"); + request.addParameter("address", "27bi7CD8d94AgXY3XFS9A9vx78Si5MqrECz"); getRewardServlet.doPost(request, response); try { String contentAsString = response.getContentAsString(); diff --git a/framework/src/test/java/org/tron/core/services/http/HttpServletTest.java b/framework/src/test/java/org/tron/core/services/http/HttpServletTest.java index dfd4c569e3e..03cf11f39a1 100644 --- a/framework/src/test/java/org/tron/core/services/http/HttpServletTest.java +++ b/framework/src/test/java/org/tron/core/services/http/HttpServletTest.java @@ -107,6 +107,7 @@ public class HttpServletTest { private ListNodesServlet listNodesServlet; private ListProposalsServlet listProposalsServlet; private ListWitnessesServlet listWitnessesServlet; + private GetPaginatedNowWitnessListServlet getPaginatedNowWitnessListServlet; private MarketCancelOrderServlet marketCancelOrderServlet; private MarketSellAssetServlet marketSellAssetServlet; private MetricsServlet metricsServlet; @@ -244,6 +245,7 @@ public void setUp() { listNodesServlet = new ListNodesServlet(); listProposalsServlet = new ListProposalsServlet(); listWitnessesServlet = new ListWitnessesServlet(); + getPaginatedNowWitnessListServlet = new GetPaginatedNowWitnessListServlet(); marketCancelOrderServlet = new MarketCancelOrderServlet(); marketSellAssetServlet = new MarketSellAssetServlet(); metricsServlet = new MetricsServlet(); @@ -367,6 +369,7 @@ public void doGetTest() { listNodesServlet.doGet(request, response); listProposalsServlet.doGet(request, response); listWitnessesServlet.doGet(request, response); + getPaginatedNowWitnessListServlet.doGet(request, response); marketCancelOrderServlet.doGet(request, response); marketSellAssetServlet.doGet(request, response); metricsServlet.doGet(request, response); @@ -499,6 +502,7 @@ public void doPostTest() { listNodesServlet.doPost(request, response); listProposalsServlet.doPost(request, response); listWitnessesServlet.doPost(request, response); + getPaginatedNowWitnessListServlet.doPost(request, response); marketCancelOrderServlet.doPost(request, response); marketSellAssetServlet.doPost(request, response); participateAssetIssueServlet.doPost(request, response); diff --git a/framework/src/test/java/org/tron/core/services/jsonrpc/BuildArgumentsTest.java b/framework/src/test/java/org/tron/core/services/jsonrpc/BuildArgumentsTest.java index f9e264c515f..952e9c81467 100644 --- a/framework/src/test/java/org/tron/core/services/jsonrpc/BuildArgumentsTest.java +++ b/framework/src/test/java/org/tron/core/services/jsonrpc/BuildArgumentsTest.java @@ -8,8 +8,8 @@ import org.tron.core.Constant; import org.tron.core.Wallet; import org.tron.core.config.args.Args; -import org.tron.core.exception.JsonRpcInvalidParamsException; -import org.tron.core.exception.JsonRpcInvalidRequestException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidRequestException; import org.tron.core.services.jsonrpc.types.BuildArguments; import org.tron.core.services.jsonrpc.types.CallArguments; import org.tron.protos.Protocol; diff --git a/framework/src/test/java/org/tron/core/services/jsonrpc/CallArgumentsTest.java b/framework/src/test/java/org/tron/core/services/jsonrpc/CallArgumentsTest.java index 19dd76e5e07..1d7f568453b 100644 --- a/framework/src/test/java/org/tron/core/services/jsonrpc/CallArgumentsTest.java +++ b/framework/src/test/java/org/tron/core/services/jsonrpc/CallArgumentsTest.java @@ -8,8 +8,8 @@ import org.tron.core.Constant; import org.tron.core.Wallet; import org.tron.core.config.args.Args; -import org.tron.core.exception.JsonRpcInvalidParamsException; -import org.tron.core.exception.JsonRpcInvalidRequestException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidRequestException; import org.tron.core.services.jsonrpc.types.CallArguments; import org.tron.protos.Protocol; diff --git a/framework/src/test/java/org/tron/core/services/jsonrpc/JsonRpcErrorResolverTest.java b/framework/src/test/java/org/tron/core/services/jsonrpc/JsonRpcErrorResolverTest.java new file mode 100644 index 00000000000..d8e64308ab8 --- /dev/null +++ b/framework/src/test/java/org/tron/core/services/jsonrpc/JsonRpcErrorResolverTest.java @@ -0,0 +1,75 @@ +package org.tron.core.services.jsonrpc; + +import com.fasterxml.jackson.databind.JsonNode; +import com.googlecode.jsonrpc4j.ErrorData; +import com.googlecode.jsonrpc4j.ErrorResolver.JsonError; +import com.googlecode.jsonrpc4j.JsonRpcError; +import com.googlecode.jsonrpc4j.JsonRpcErrors; +import java.lang.reflect.Method; +import java.util.ArrayList; +import java.util.List; +import org.junit.Assert; +import org.junit.Test; +import org.tron.core.exception.jsonrpc.JsonRpcException; +import org.tron.core.exception.jsonrpc.JsonRpcInternalException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidParamsException; +import org.tron.core.exception.jsonrpc.JsonRpcInvalidRequestException; + +public class JsonRpcErrorResolverTest { + + private final JsonRpcErrorResolver resolver = JsonRpcErrorResolver.INSTANCE; + + @JsonRpcErrors({ + @JsonRpcError(exception = JsonRpcInvalidRequestException.class, code = -32600, data = "{}"), + @JsonRpcError(exception = JsonRpcInvalidParamsException.class, code = -32602, data = "{}"), + @JsonRpcError(exception = JsonRpcInternalException.class, code = -32000, data = "{}"), + @JsonRpcError(exception = JsonRpcException.class, code = -1) + }) + public void dummyMethod() { + } + + @Test + public void testResolveErrorWithTronException() throws Exception { + + String message = "JsonRpcInvalidRequestException"; + + JsonRpcException exception = new JsonRpcInvalidRequestException(message); + Method method = this.getClass().getMethod("dummyMethod"); + List arguments = new ArrayList<>(); + + JsonError error = resolver.resolveError(exception, method, arguments); + Assert.assertNotNull(error); + Assert.assertEquals(-32600, error.code); + Assert.assertEquals(message, error.message); + Assert.assertEquals("{}", error.data); + + message = "JsonRpcInternalException"; + String data = "JsonRpcInternalException data"; + exception = new JsonRpcInternalException(message, data); + error = resolver.resolveError(exception, method, arguments); + + Assert.assertNotNull(error); + Assert.assertEquals(-32000, error.code); + Assert.assertEquals(message, error.message); + Assert.assertEquals(data, error.data); + + exception = new JsonRpcInternalException(message, null); + error = resolver.resolveError(exception, method, arguments); + + Assert.assertNotNull(error); + Assert.assertEquals(-32000, error.code); + Assert.assertEquals(message, error.message); + Assert.assertEquals("{}", error.data); + + message = "JsonRpcException"; + exception = new JsonRpcException(message, null); + error = resolver.resolveError(exception, method, arguments); + + Assert.assertNotNull(error); + Assert.assertEquals(-1, error.code); + Assert.assertEquals(message, error.message); + Assert.assertTrue(error.data instanceof ErrorData); + + } + +} \ No newline at end of file diff --git a/framework/src/test/java/org/tron/core/services/jsonrpc/TransactionReceiptTest.java b/framework/src/test/java/org/tron/core/services/jsonrpc/TransactionReceiptTest.java index f10526e30a4..23bc11e293f 100644 --- a/framework/src/test/java/org/tron/core/services/jsonrpc/TransactionReceiptTest.java +++ b/framework/src/test/java/org/tron/core/services/jsonrpc/TransactionReceiptTest.java @@ -8,31 +8,32 @@ import org.tron.common.utils.ByteArray; import org.tron.core.Constant; import org.tron.core.Wallet; +import org.tron.core.capsule.BlockCapsule; import org.tron.core.capsule.TransactionRetCapsule; import org.tron.core.config.args.Args; +import org.tron.core.exception.jsonrpc.JsonRpcInternalException; import org.tron.core.services.jsonrpc.types.TransactionReceipt; +import org.tron.core.services.jsonrpc.types.TransactionReceipt.TransactionContext; import org.tron.core.store.TransactionRetStore; import org.tron.protos.Protocol; public class TransactionReceiptTest extends BaseTest { - @Resource - private Wallet wallet; + @Resource private Wallet wallet; - @Resource - private TransactionRetStore transactionRetStore; + @Resource private TransactionRetStore transactionRetStore; static { - Args.setParam(new String[]{"-d", dbPath()}, Constant.TEST_CONF); + Args.setParam(new String[] {"-d", dbPath()}, Constant.TEST_CONF); } @Test - public void testTransactionReceipt() { + public void testTransactionReceipt() throws JsonRpcInternalException { Protocol.TransactionInfo transactionInfo = Protocol.TransactionInfo.newBuilder() .setId(ByteString.copyFrom("1".getBytes())) .setContractAddress(ByteString.copyFrom("address1".getBytes())) .setReceipt(Protocol.ResourceReceipt.newBuilder() - .setEnergyUsageTotal(0L) + .setEnergyUsageTotal(100L) .setResult(Protocol.Transaction.Result.contractResult.DEFAULT) .build()) .addLog(Protocol.TransactionInfo.Log.newBuilder() @@ -50,17 +51,49 @@ public void testTransactionReceipt() { raw.addContract(contract.build()); Protocol.Transaction transaction = Protocol.Transaction.newBuilder().setRawData(raw).build(); - TransactionReceipt transactionReceipt = new TransactionReceipt( - Protocol.Block.newBuilder().setBlockHeader( - Protocol.BlockHeader.newBuilder().setRawData( - Protocol.BlockHeader.raw.newBuilder().setNumber(1))).addTransactions( - transaction).build(), transactionInfo, wallet); + Protocol.Block block = Protocol.Block.newBuilder().setBlockHeader( + Protocol.BlockHeader.newBuilder().setRawData( + Protocol.BlockHeader.raw.newBuilder().setNumber(1))).addTransactions( + transaction).build(); - Assert.assertEquals(transactionReceipt.getBlockNumber(),"0x1"); - Assert.assertEquals(transactionReceipt.getTransactionIndex(),"0x0"); - Assert.assertEquals(transactionReceipt.getLogs().length,1); - Assert.assertEquals(transactionReceipt.getBlockHash(), - "0x0000000000000001464f071c8a336fd22eb5145dff1b245bda013ec89add8497"); - } + BlockCapsule blockCapsule = new BlockCapsule(block); + long energyFee = wallet.getEnergyFee(blockCapsule.getTimeStamp()); + TransactionReceipt.TransactionContext context + = new TransactionContext(0, 2, 3); + + TransactionReceipt transactionReceipt = + new TransactionReceipt(blockCapsule, transactionInfo, context, energyFee); + + Assert.assertNotNull(transactionReceipt); + String blockHash = "0x0000000000000001464f071c8a336fd22eb5145dff1b245bda013ec89add8497"; + + // assert basic fields + Assert.assertEquals(transactionReceipt.getBlockHash(), blockHash); + Assert.assertEquals(transactionReceipt.getBlockNumber(), "0x1"); + Assert.assertEquals(transactionReceipt.getTransactionHash(), "0x31"); + Assert.assertEquals(transactionReceipt.getTransactionIndex(), "0x0"); + Assert.assertEquals(transactionReceipt.getCumulativeGasUsed(), ByteArray.toJsonHex(102)); + Assert.assertEquals(transactionReceipt.getGasUsed(), ByteArray.toJsonHex(100)); + Assert.assertEquals(transactionReceipt.getEffectiveGasPrice(), ByteArray.toJsonHex(energyFee)); + Assert.assertEquals(transactionReceipt.getStatus(), "0x1"); + // assert contract fields + Assert.assertEquals(transactionReceipt.getFrom(), ByteArray.toJsonHexAddress(new byte[0])); + Assert.assertEquals(transactionReceipt.getTo(), ByteArray.toJsonHexAddress(new byte[0])); + Assert.assertNull(transactionReceipt.getContractAddress()); + + // assert logs fields + Assert.assertEquals(transactionReceipt.getLogs().length, 1); + Assert.assertEquals(transactionReceipt.getLogs()[0].getLogIndex(), "0x3"); + Assert.assertEquals( + transactionReceipt.getLogs()[0].getBlockHash(), blockHash); + Assert.assertEquals(transactionReceipt.getLogs()[0].getBlockNumber(), "0x1"); + Assert.assertEquals(transactionReceipt.getLogs()[0].getTransactionHash(), "0x31"); + Assert.assertEquals(transactionReceipt.getLogs()[0].getTransactionIndex(), "0x0"); + + // assert default fields + Assert.assertNull(transactionReceipt.getRoot()); + Assert.assertEquals(transactionReceipt.getType(), "0x0"); + Assert.assertEquals(transactionReceipt.getLogsBloom(), ByteArray.toJsonHex(new byte[256])); + } } diff --git a/framework/src/test/java/org/tron/core/services/jsonrpc/TransactionResultTest.java b/framework/src/test/java/org/tron/core/services/jsonrpc/TransactionResultTest.java index a71441c73b4..625981df3bb 100644 --- a/framework/src/test/java/org/tron/core/services/jsonrpc/TransactionResultTest.java +++ b/framework/src/test/java/org/tron/core/services/jsonrpc/TransactionResultTest.java @@ -9,49 +9,61 @@ import org.tron.core.Constant; import org.tron.core.Wallet; import org.tron.core.capsule.BlockCapsule; +import org.tron.core.capsule.TransactionCapsule; import org.tron.core.config.args.Args; import org.tron.core.services.jsonrpc.types.TransactionResult; import org.tron.protos.Protocol; +import org.tron.protos.contract.SmartContractOuterClass; public class TransactionResultTest extends BaseTest { @Resource private Wallet wallet; + private static final String OWNER_ADDRESS = "41548794500882809695a8a687866e76d4271a1abc"; + private static final String CONTRACT_ADDRESS = "A0B4750E2CD76E19DCA331BF5D089B71C3C2798548"; + static { - Args.setParam(new String[]{"-d", dbPath()}, Constant.TEST_CONF); + Args.setParam(new String[] {"-d", dbPath()}, Constant.TEST_CONF); } @Test public void testBuildTransactionResultWithBlock() { - Protocol.Transaction.raw.Builder raw = Protocol.Transaction.raw.newBuilder().addContract( - Protocol.Transaction.Contract.newBuilder().setType( - Protocol.Transaction.Contract.ContractType.TriggerSmartContract)); - Protocol.Transaction transaction = Protocol.Transaction.newBuilder().setRawData(raw).build(); + SmartContractOuterClass.TriggerSmartContract.Builder builder2 = + SmartContractOuterClass.TriggerSmartContract.newBuilder() + .setOwnerAddress(ByteString.copyFrom(ByteArray.fromHexString(OWNER_ADDRESS))) + .setContractAddress(ByteString.copyFrom(ByteArray.fromHexString(CONTRACT_ADDRESS))); + TransactionCapsule transactionCapsule = new TransactionCapsule(builder2.build(), + Protocol.Transaction.Contract.ContractType.TriggerSmartContract); + Protocol.Transaction transaction = transactionCapsule.getInstance(); BlockCapsule blockCapsule = new BlockCapsule(Protocol.Block.newBuilder().setBlockHeader( Protocol.BlockHeader.newBuilder().setRawData(Protocol.BlockHeader.raw.newBuilder() .setParentHash(ByteString.copyFrom(ByteArray.fromHexString( "0304f784e4e7bae517bcab94c3e0c9214fb4ac7ff9d7d5a937d1f40031f87b82"))) .setNumber(9))).addTransactions(transaction).build()); - TransactionResult transactionResult = new TransactionResult(blockCapsule,0, transaction, - 100,1, wallet); + TransactionResult transactionResult = new TransactionResult(blockCapsule, 0, transaction, + 100, 1, wallet); Assert.assertEquals(transactionResult.getBlockNumber(), "0x9"); - Assert.assertEquals(transactionResult.getHash(), - "0xdebef90d0a8077620711b1b5af2b702665887ddcbf80868108026e1ab5e0bfb7"); + Assert.assertEquals("0x5691531881bc44adbc722060d85fdf29265823db8e884b0d104fcfbba253cf11", + transactionResult.getHash()); Assert.assertEquals(transactionResult.getGasPrice(), "0x1"); Assert.assertEquals(transactionResult.getGas(), "0x64"); } @Test public void testBuildTransactionResult() { - Protocol.Transaction.raw.Builder raw = Protocol.Transaction.raw.newBuilder().addContract( - Protocol.Transaction.Contract.newBuilder().setType( - Protocol.Transaction.Contract.ContractType.TriggerSmartContract)); - Protocol.Transaction transaction = Protocol.Transaction.newBuilder().setRawData(raw).build(); - TransactionResult transactionResult = new TransactionResult(transaction, wallet); - Assert.assertEquals(transactionResult.getHash(), - "0xdebef90d0a8077620711b1b5af2b702665887ddcbf80868108026e1ab5e0bfb7"); + SmartContractOuterClass.TriggerSmartContract.Builder builder2 = + SmartContractOuterClass.TriggerSmartContract.newBuilder() + .setOwnerAddress(ByteString.copyFrom(ByteArray.fromHexString(OWNER_ADDRESS))) + .setContractAddress(ByteString.copyFrom(ByteArray.fromHexString(CONTRACT_ADDRESS))); + TransactionCapsule transactionCapsule = new TransactionCapsule(builder2.build(), + Protocol.Transaction.Contract.ContractType.TriggerSmartContract); + + TransactionResult transactionResult = + new TransactionResult(transactionCapsule.getInstance(), wallet); + Assert.assertEquals("0x5691531881bc44adbc722060d85fdf29265823db8e884b0d104fcfbba253cf11", + transactionResult.getHash()); Assert.assertEquals(transactionResult.getGasPrice(), "0x"); Assert.assertEquals(transactionResult.getNonce(), "0x0000000000000000"); } diff --git a/framework/src/test/java/org/tron/core/services/stop/BlockTimeStopTest.java b/framework/src/test/java/org/tron/core/services/stop/BlockTimeStopTest.java index 1e16ad6c3b0..bf7bcf2cd7e 100644 --- a/framework/src/test/java/org/tron/core/services/stop/BlockTimeStopTest.java +++ b/framework/src/test/java/org/tron/core/services/stop/BlockTimeStopTest.java @@ -1,8 +1,10 @@ package org.tron.core.services.stop; import java.text.ParseException; +import java.time.ZoneOffset; import java.time.format.DateTimeFormatter; import java.util.Date; +import java.util.TimeZone; import lombok.extern.slf4j.Slf4j; import org.junit.Assert; import org.tron.common.cron.CronExpression; @@ -12,14 +14,14 @@ public class BlockTimeStopTest extends ConditionallyStopTest { private static final DateTimeFormatter pattern = DateTimeFormatter .ofPattern("ss mm HH dd MM ? yyyy"); - private static final String time = localDateTime.plusSeconds(12 * 3).format(pattern); private static CronExpression cronExpression; static { try { - cronExpression = new CronExpression(time); + cronExpression = new CronExpression(localDateTime.plusSeconds(12 * 3).format(pattern)); + cronExpression.setTimeZone(TimeZone.getTimeZone(ZoneOffset.UTC)); } catch (ParseException e) { logger.error("{}", e.getMessage()); } diff --git a/framework/src/test/java/org/tron/core/services/stop/ConditionallyStopTest.java b/framework/src/test/java/org/tron/core/services/stop/ConditionallyStopTest.java index b417b27b380..f9795def416 100644 --- a/framework/src/test/java/org/tron/core/services/stop/ConditionallyStopTest.java +++ b/framework/src/test/java/org/tron/core/services/stop/ConditionallyStopTest.java @@ -2,13 +2,11 @@ import com.google.common.collect.Maps; import com.google.protobuf.ByteString; -import java.io.File; import java.io.IOException; +import java.time.Instant; import java.time.LocalDateTime; -import java.time.ZoneId; -import java.time.ZonedDateTime; +import java.time.ZoneOffset; import java.util.ArrayList; -import java.util.Arrays; import java.util.List; import java.util.Map; import java.util.concurrent.atomic.AtomicInteger; @@ -16,7 +14,6 @@ import java.util.stream.IntStream; import lombok.extern.slf4j.Slf4j; import org.junit.After; -import org.junit.Assert; import org.junit.Before; import org.junit.ClassRule; import org.junit.Test; @@ -25,11 +22,10 @@ import org.tron.common.crypto.ECKey; import org.tron.common.parameter.CommonParameter; import org.tron.common.utils.ByteArray; -import org.tron.common.utils.FileUtil; -import org.tron.common.utils.LocalWitnesses; -import org.tron.common.utils.PublicMethod; import org.tron.common.utils.Sha256Hash; import org.tron.common.utils.Utils; +import org.tron.consensus.ConsensusDelegate; +import org.tron.consensus.dpos.DposService; import org.tron.consensus.dpos.DposSlot; import org.tron.core.ChainBaseManager; import org.tron.core.Constant; @@ -39,23 +35,18 @@ import org.tron.core.config.DefaultConfig; import org.tron.core.config.args.Args; import org.tron.core.consensus.ConsensusService; -import org.tron.core.db.BlockGenerate; import org.tron.core.db.Manager; import org.tron.core.net.TronNetDelegate; import org.tron.protos.Protocol; -@Slf4j -public abstract class ConditionallyStopTest extends BlockGenerate { +@Slf4j(topic = "test") +public abstract class ConditionallyStopTest { @ClassRule public static final TemporaryFolder temporaryFolder = new TemporaryFolder(); static ChainBaseManager chainManager; private static DposSlot dposSlot; - - private final String key = PublicMethod.getRandomPrivateKey(); - private final byte[] privateKey = ByteArray.fromHexString(key); - private final AtomicInteger port = new AtomicInteger(0); protected String dbPath; protected Manager dbManager; @@ -63,10 +54,12 @@ public abstract class ConditionallyStopTest extends BlockGenerate { private TronNetDelegate tronNetDelegate; private TronApplicationContext context; + private DposService dposService; + private ConsensusDelegate consensusDelegate; - static LocalDateTime localDateTime = LocalDateTime.now(); - private long time = ZonedDateTime.of(localDateTime, - ZoneId.systemDefault()).toInstant().toEpochMilli(); + private static final Instant instant = Instant.parse("2025-10-01T00:00:00Z"); + private final long time = instant.toEpochMilli(); + static LocalDateTime localDateTime = LocalDateTime.ofInstant(instant, ZoneOffset.UTC); protected abstract void initParameter(CommonParameter parameter); @@ -76,6 +69,8 @@ protected void initDbPath() throws IOException { dbPath = temporaryFolder.newFolder().toString(); } + private Map witnesses; + @Before public void init() throws Exception { @@ -84,30 +79,37 @@ public void init() throws Exception { logger.info("Full node running."); Args.setParam(new String[] {"-d", dbPath}, Constant.TEST_CONF); Args.getInstance().setNodeListenPort(10000 + port.incrementAndGet()); - + Args.getInstance().genesisBlock.setTimestamp(Long.toString(time)); initParameter(Args.getInstance()); context = new TronApplicationContext(DefaultConfig.class); dbManager = context.getBean(Manager.class); - setManager(dbManager); dposSlot = context.getBean(DposSlot.class); ConsensusService consensusService = context.getBean(ConsensusService.class); consensusService.start(); chainManager = dbManager.getChainBaseManager(); tronNetDelegate = context.getBean(TronNetDelegate.class); + dposService = context.getBean(DposService.class); + consensusDelegate = context.getBean(ConsensusDelegate.class); tronNetDelegate.setExit(false); currentHeader = dbManager.getDynamicPropertiesStore() .getLatestBlockHeaderNumberFromDB(); - byte[] address = PublicMethod.getAddressByteByPrivateKey(key); - ByteString addressByte = ByteString.copyFrom(address); - WitnessCapsule witnessCapsule = new WitnessCapsule(addressByte); - chainManager.getWitnessStore().put(addressByte.toByteArray(), witnessCapsule); - chainManager.addWitness(addressByte); - - AccountCapsule accountCapsule = - new AccountCapsule(Protocol.Account.newBuilder().setAddress(addressByte).build()); - chainManager.getAccountStore().put(addressByte.toByteArray(), accountCapsule); + chainManager.getWitnessScheduleStore().reset(); + chainManager.getWitnessStore().reset(); + witnesses = addTestWitnessAndAccount(); + + List allWitnesses = new ArrayList<>(); + consensusDelegate.getAllWitnesses().forEach(witnessCapsule -> + allWitnesses.add(witnessCapsule.getAddress())); + dposService.updateWitness(allWitnesses); + List activeWitnesses = consensusDelegate.getActiveWitnesses(); + activeWitnesses.forEach(address -> { + WitnessCapsule witnessCapsule = consensusDelegate.getWitness(address.toByteArray()); + witnessCapsule.setIsJobs(true); + consensusDelegate.saveWitness(witnessCapsule); + }); + chainManager.getDynamicPropertiesStore().saveNextMaintenanceTime(time); } @After @@ -116,72 +118,57 @@ public void destroy() { context.destroy(); } - private void generateBlock(Map witnessAndAccount) throws Exception { + private void generateBlock() throws Exception { BlockCapsule block = createTestBlockCapsule( - chainManager.getDynamicPropertiesStore().getLatestBlockHeaderTimestamp() + 3000, + chainManager.getNextBlockSlotTime(), chainManager.getDynamicPropertiesStore().getLatestBlockHeaderNumber() + 1, - chainManager.getDynamicPropertiesStore().getLatestBlockHeaderHash().getByteString(), - witnessAndAccount); + chainManager.getDynamicPropertiesStore().getLatestBlockHeaderHash().getByteString()); tronNetDelegate.processBlock(block, false); + + logger.info("headerNum: {} solidityNum: {}, dbNum: {}", + block.getNum(), chainManager.getDynamicPropertiesStore().getLatestSolidifiedBlockNum(), + chainManager.getDynamicPropertiesStore().getLatestBlockHeaderNumberFromDB()); } - @Test + @Test(timeout = 30_000) // milliseconds public void testStop() throws Exception { - final ECKey ecKey = ECKey.fromPrivate(privateKey); - Assert.assertNotNull(ecKey); - byte[] address = ecKey.getAddress(); - WitnessCapsule witnessCapsule = new WitnessCapsule(ByteString.copyFrom(address)); - chainManager.getWitnessScheduleStore().saveActiveWitnesses(new ArrayList<>()); - chainManager.addWitness(ByteString.copyFrom(address)); - - Protocol.Block block = getSignedBlock(witnessCapsule.getAddress(), time, privateKey); - - tronNetDelegate.processBlock(new BlockCapsule(block), false); - - Map witnessAndAccount = addTestWitnessAndAccount(); - witnessAndAccount.put(ByteString.copyFrom(address), key); while (!tronNetDelegate.isHitDown()) { - generateBlock(witnessAndAccount); + generateBlock(); } - Assert.assertTrue(tronNetDelegate.isHitDown()); check(); } - private Map addTestWitnessAndAccount() { - chainManager.getWitnesses().clear(); - return IntStream.range(0, 2) + private Map addTestWitnessAndAccount() { + return IntStream.range(0, 27) .mapToObj( i -> { ECKey ecKey = new ECKey(Utils.getRandom()); String privateKey = ByteArray.toHexString(ecKey.getPrivKey().toByteArray()); ByteString address = ByteString.copyFrom(ecKey.getAddress()); - WitnessCapsule witnessCapsule = new WitnessCapsule(address); + WitnessCapsule witnessCapsule = new WitnessCapsule(address, 27 - i, "SR" + i); chainManager.getWitnessStore().put(address.toByteArray(), witnessCapsule); - chainManager.addWitness(address); - AccountCapsule accountCapsule = new AccountCapsule(Protocol.Account.newBuilder().setAddress(address).build()); chainManager.getAccountStore().put(address.toByteArray(), accountCapsule); - return Maps.immutableEntry(address, privateKey); + return Maps.immutableEntry(ByteArray.toHexString(ecKey.getAddress()), privateKey); }) .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue)); } private BlockCapsule createTestBlockCapsule(long time, - long number, ByteString hash, - Map witnessAddressMap) { - ByteString witnessAddress = dposSlot.getScheduledWitness(dposSlot.getSlot(time)); - BlockCapsule blockCapsule = new BlockCapsule(number, Sha256Hash.wrap(hash), time, - witnessAddress); + long number, ByteString hash) { + long slot = dposSlot.getSlot(time); + ByteString witness = dposSlot.getScheduledWitness(slot); + BlockCapsule blockCapsule = new BlockCapsule(number, Sha256Hash.wrap(hash), time, witness); blockCapsule.generatedByMyself = true; blockCapsule.setMerkleRoot(); - blockCapsule.sign(ByteArray.fromHexString(witnessAddressMap.get(witnessAddress))); + String pri = witnesses.get(ByteArray.toHexString(witness.toByteArray())); + blockCapsule.sign(ByteArray.fromHexString(pri)); return blockCapsule; } - } diff --git a/framework/src/test/java/org/tron/core/zksnark/LibrustzcashTest.java b/framework/src/test/java/org/tron/core/zksnark/LibrustzcashTest.java index 049fb2528b1..5d403b54f90 100644 --- a/framework/src/test/java/org/tron/core/zksnark/LibrustzcashTest.java +++ b/framework/src/test/java/org/tron/core/zksnark/LibrustzcashTest.java @@ -84,7 +84,8 @@ public static void init() { }, "config-test-mainnet.conf" ); - Args.setFullNodeAllowShieldedTransaction(true); + Args.getInstance().setAllowShieldedTransactionApi(true); + ZksnarkInitService.librustzcashInitZksnarkParams(); } private static int randomInt(int minInt, int maxInt) { @@ -115,10 +116,6 @@ public static void test(byte[] K, byte[] ovk, byte[] cv, byte[] cm, byte[] epk) null, 0, cipher_nonce, K))); } - public static void librustzcashInitZksnarkParams() { - ZksnarkInitService.librustzcashInitZksnarkParams(); - } - @Test public void testLibsodium() throws ZksnarkException { byte[] K = new byte[32]; @@ -275,7 +272,6 @@ public long benchmarkCreateSpend() throws ZksnarkException { @Ignore @Test public void calBenchmarkSpendConcurrent() throws Exception { - librustzcashInitZksnarkParams(); System.out.println("--- load ok ---"); int count = 2; @@ -301,13 +297,13 @@ public void calBenchmarkSpendConcurrent() throws Exception { })); countDownLatch.await(); + generatePool.shutdown(); logger.info("generate cost time:" + (System.currentTimeMillis() - startGenerate)); } @Test public void calBenchmarkSpend() throws ZksnarkException { - librustzcashInitZksnarkParams(); System.out.println("--- load ok ---"); int count = 2; @@ -374,7 +370,6 @@ public long benchmarkCreateSaplingSpend() throws BadItemException, ZksnarkExcept @Test public void calBenchmarkCreateSaplingSpend() throws BadItemException, ZksnarkException { - librustzcashInitZksnarkParams(); System.out.println("--- load ok ---"); int count = 2; @@ -451,7 +446,6 @@ public long benchmarkCreateSaplingOutput() throws BadItemException, ZksnarkExcep @Test public void calBenchmarkCreateSaplingOutPut() throws BadItemException, ZksnarkException { - librustzcashInitZksnarkParams(); System.out.println("--- load ok ---"); int count = 2; @@ -479,7 +473,6 @@ public void calBenchmarkCreateSaplingOutPut() throws BadItemException, ZksnarkEx @Test public void checkVerifyOutErr() throws ZksnarkException { - librustzcashInitZksnarkParams(); System.out.println("--- load ok ---"); // expect fail diff --git a/framework/src/test/java/org/tron/core/zksnark/SaplingNoteTest.java b/framework/src/test/java/org/tron/core/zksnark/SaplingNoteTest.java index da4df70d9ac..34913c98ccc 100644 --- a/framework/src/test/java/org/tron/core/zksnark/SaplingNoteTest.java +++ b/framework/src/test/java/org/tron/core/zksnark/SaplingNoteTest.java @@ -22,7 +22,7 @@ public class SaplingNoteTest { @BeforeClass public static void init() { - Args.setFullNodeAllowShieldedTransaction(true); + Args.getInstance().setAllowShieldedTransactionApi(true); // Args.getInstance().setAllowShieldedTransaction(1); ZksnarkInitService.librustzcashInitZksnarkParams(); } diff --git a/framework/src/test/java/org/tron/core/zksnark/SendCoinShieldTest.java b/framework/src/test/java/org/tron/core/zksnark/SendCoinShieldTest.java index 7746066abfa..e7dfa06d094 100644 --- a/framework/src/test/java/org/tron/core/zksnark/SendCoinShieldTest.java +++ b/framework/src/test/java/org/tron/core/zksnark/SendCoinShieldTest.java @@ -17,6 +17,7 @@ import lombok.extern.slf4j.Slf4j; import org.junit.Assert; import org.junit.Before; +import org.junit.BeforeClass; import org.junit.Ignore; import org.junit.Test; import org.tron.api.GrpcAPI; @@ -118,6 +119,11 @@ public class SendCoinShieldTest extends BaseTest { .fromHexString("030c8c2bc59fb3eb8afb047a8ea4b028743d23e7d38c6fa30908358431e2314d"); } + @BeforeClass + public static void initZksnarkParams() { + ZksnarkInitService.librustzcashInitZksnarkParams(); + } + /** * Init data. */ @@ -219,10 +225,6 @@ private IncrementalMerkleVoucherContainer createSimpleMerkleVoucherContainer(byt return tree.toVoucher(); } - private void librustzcashInitZksnarkParams() { - ZksnarkInitService.librustzcashInitZksnarkParams(); - } - @Test public void testStringRevert() { byte[] bytes = ByteArray @@ -235,7 +237,6 @@ public void testStringRevert() { @Test public void testGenerateSpendProof() throws Exception { - librustzcashInitZksnarkParams(); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); SpendingKey sk = SpendingKey .decode("ff2c06269315333a9207f817d2eca0ac555ca8f90196976324c7756504e7c9ee"); @@ -270,7 +271,6 @@ public void testGenerateSpendProof() throws Exception { @Test public void generateOutputProof() throws ZksnarkException { - librustzcashInitZksnarkParams(); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); SpendingKey spendingKey = SpendingKey.random(); FullViewingKey fullViewingKey = spendingKey.fullViewingKey(); @@ -289,7 +289,6 @@ public void generateOutputProof() throws ZksnarkException { @Test public void verifyOutputProof() throws ZksnarkException { - librustzcashInitZksnarkParams(); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); SpendingKey spendingKey = SpendingKey.random(); FullViewingKey fullViewingKey = spendingKey.fullViewingKey(); @@ -321,7 +320,6 @@ public void verifyOutputProof() throws ZksnarkException { @Test public void testDecryptReceiveWithIvk() throws ZksnarkException { //verify c_enc - librustzcashInitZksnarkParams(); ZenTransactionBuilder builder = new ZenTransactionBuilder(); SpendingKey spendingKey = SpendingKey.random(); @@ -385,9 +383,6 @@ public String byte2intstring(byte[] input) { @Test public void testDecryptReceiveWithOvk() throws Exception { - //decode c_out with ovk. - librustzcashInitZksnarkParams(); - // construct payment address SpendingKey spendingKey2 = SpendingKey .decode("ff2c06269315333a9207f817d2eca0ac555ca8f90196976324c7756504e7c9ee"); @@ -467,7 +462,6 @@ public void pushShieldedTransactionAndDecryptWithIvk() AccountResourceInsufficientException, InvalidProtocolBufferException, ZksnarkException { long ctx = JLibrustzcash.librustzcashSaplingProvingCtxInit(); - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); dbManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(1000 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -556,7 +550,6 @@ public void pushShieldedTransactionAndDecryptWithOvk() AccountResourceInsufficientException, InvalidProtocolBufferException, ZksnarkException { long ctx = JLibrustzcash.librustzcashSaplingProvingCtxInit(); - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); dbManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(1000 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -643,10 +636,8 @@ private byte[] getHash() { @Ignore @Test public void checkZksnark() throws BadItemException, ZksnarkException { - librustzcashInitZksnarkParams(); long ctx = JLibrustzcash.librustzcashSaplingProvingCtxInit(); // generate spend proof - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); dbManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(4010 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -679,7 +670,6 @@ public void checkZksnark() throws BadItemException, ZksnarkException { @Test public void testVerifySpendProof() throws BadItemException, ZksnarkException { - librustzcashInitZksnarkParams(); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); SpendingKey sk = SpendingKey .decode("ff2c06269315333a9207f817d2eca0ac555ca8f90196976324c7756504e7c9ee"); @@ -717,7 +707,6 @@ public void testVerifySpendProof() throws BadItemException, ZksnarkException { public void saplingBindingSig() throws BadItemException, ZksnarkException { long ctx = JLibrustzcash.librustzcashSaplingProvingCtxInit(); // generate spend proof - librustzcashInitZksnarkParams(); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); SpendingKey sk = SpendingKey .decode("ff2c06269315333a9207f817d2eca0ac555ca8f90196976324c7756504e7c9ee"); @@ -756,7 +745,6 @@ public void pushShieldedTransaction() ContractExeException, AccountResourceInsufficientException, ZksnarkException { long ctx = JLibrustzcash.librustzcashSaplingProvingCtxInit(); // generate spend proof - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); dbManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(4010 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -789,7 +777,6 @@ public void pushShieldedTransaction() @Test public void finalCheck() throws BadItemException, ZksnarkException { long ctx = JLibrustzcash.librustzcashSaplingProvingCtxInit(); - librustzcashInitZksnarkParams(); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); // generate spend proof SpendingKey sk = SpendingKey @@ -965,7 +952,6 @@ public void getSpendingKey() throws Exception { @Test public void testTwoCMWithDiffSkInOneTx() throws Exception { // generate spend proof - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(110 * 1000000L); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1025,7 +1011,6 @@ private void executeTx(TransactionCapsule transactionCap) throws Exception { @Test public void testValueBalance() throws Exception { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); //case 1, a public input, no input cm, an output cm, no public output { @@ -1246,7 +1231,6 @@ public void testValueBalance() throws Exception { @Test public void TestCreateMultipleTxAtTheSameTime() throws Exception { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); List txList = Lists.newArrayList(); //case 1, a public input, no input cm, an output cm, no public output @@ -1364,7 +1348,6 @@ public void TestCreateMultipleTxAtTheSameTime() throws Exception { @Test public void TestCtxGeneratesTooMuchProof() throws Exception { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); //case 3, no public input, an input cm, no output cm, a public output { @@ -1439,7 +1422,6 @@ public SpendDescriptionCapsule generateSpendProof(SpendDescriptionInfo spend, lo @Test public void TestGeneratesProofWithDiffCtx() throws Exception { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); //case 3, no public input, an input cm, no output cm, a public output @@ -1498,7 +1480,6 @@ public SpendDescriptionCapsule generateSpendProof(SpendDescriptionInfo spend, lo @Test public void TestGeneratesProofWithWrongAlpha() throws Exception { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); //case 3, no public input, an input cm, no output cm, a public output { @@ -1536,7 +1517,6 @@ public void TestGeneratesProofWithWrongAlpha() throws Exception { @Test public void TestGeneratesProofWithWrongRcm() throws Exception { long ctx = JLibrustzcash.librustzcashSaplingProvingCtxInit(); - librustzcashInitZksnarkParams(); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); // generate spend proof SpendingKey sk = SpendingKey.random(); @@ -1557,7 +1537,6 @@ public void TestGeneratesProofWithWrongRcm() throws Exception { @Test public void TestWrongAsk() throws Exception { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); //case 3, no public input, an input cm, no output cm, a public output @@ -1667,7 +1646,6 @@ private TransactionCapsule generateDefaultBuilder(ZenTransactionBuilder builder) @Test public void TestDefaultBuilder() throws Exception { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); dbManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(1000 * 1000000L); @@ -1678,7 +1656,6 @@ public void TestDefaultBuilder() throws Exception { @Test public void TestWrongSpendRk() throws Exception { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); { @@ -1716,7 +1693,6 @@ public SpendDescriptionCapsule generateSpendProof(SpendDescriptionInfo spend, lo @Test public void TestWrongSpendProof() throws Exception { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); { @@ -1761,7 +1737,6 @@ public SpendDescriptionCapsule generateSpendProof(SpendDescriptionInfo spend, lo @Test public void TestWrongNf() throws Exception { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); { @@ -1800,7 +1775,6 @@ public SpendDescriptionCapsule generateSpendProof(SpendDescriptionInfo spend, lo @Test public void TestWrongAnchor() throws Exception { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); { ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet) { diff --git a/framework/src/test/java/org/tron/core/zksnark/ShieldedReceiveTest.java b/framework/src/test/java/org/tron/core/zksnark/ShieldedReceiveTest.java index 2a7545f7a9b..118e0e1f384 100755 --- a/framework/src/test/java/org/tron/core/zksnark/ShieldedReceiveTest.java +++ b/framework/src/test/java/org/tron/core/zksnark/ShieldedReceiveTest.java @@ -17,6 +17,7 @@ import lombok.extern.slf4j.Slf4j; import org.junit.Assert; import org.junit.Before; +import org.junit.BeforeClass; import org.junit.Test; import org.tron.api.GrpcAPI.BytesMessage; import org.tron.api.GrpcAPI.DecryptNotes; @@ -127,9 +128,14 @@ public class ShieldedReceiveTest extends BaseTest { private static boolean init; static { - Args.setParam(new String[]{"--output-directory", dbPath()}, "config-localtest.conf"); + Args.setParam(new String[]{"--output-directory", dbPath(), "-w"}, "config-localtest.conf"); ADDRESS_ONE_PRIVATE_KEY = getRandomPrivateKey(); - FROM_ADDRESS = getHexAddressByPrivateKey(ADDRESS_ONE_PRIVATE_KEY);; + FROM_ADDRESS = getHexAddressByPrivateKey(ADDRESS_ONE_PRIVATE_KEY); + } + + @BeforeClass + public static void initZksnarkParams() { + ZksnarkInitService.librustzcashInitZksnarkParams(); } /** @@ -145,10 +151,6 @@ public void init() { init = true; } - private static void librustzcashInitZksnarkParams() { - ZksnarkInitService.librustzcashInitZksnarkParams(); - } - private static byte[] randomUint256() { return org.tron.keystore.Wallet.generateRandomBytes(32); } @@ -226,6 +228,11 @@ private void updateTotalShieldedPoolValue(long valueBalance) { chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(totalShieldedPoolValue); } + @Test + public void testIsMining() { + Assert.assertTrue(wallet.isMining()); + } + /* * test of change ShieldedTransactionFee proposal */ @@ -244,7 +251,6 @@ public void testSetShieldedTransactionFee() { */ @Test public void testCreateBeforeAllowZksnark() throws ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(0); createCapsule(); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -284,7 +290,6 @@ public void testCreateBeforeAllowZksnark() throws ZksnarkException { @Test public void testBroadcastBeforeAllowZksnark() throws ZksnarkException, SignatureFormatException, SignatureException, PermissionException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(0);// or 1 createCapsule(); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -310,20 +315,16 @@ public void testBroadcastBeforeAllowZksnark() ADDRESS_ONE_PRIVATE_KEY, chainBaseManager.getAccountStore()); try { dbManager.pushTransaction(transactionCap); - Assert.assertFalse(true); } catch (Exception e) { Assert.assertTrue(e instanceof ContractValidateException); - Assert.assertEquals( - "Not support Shielded Transaction, need to be opened by the committee", - e.getMessage()); } } /* - * generate spendproof, dataToBeSigned, outputproof example dynamically according to the params file + * generate spendproof, dataToBeSigned, + * outputproof example dynamically according to the params file */ public String[] generateSpendAndOutputParams() throws ZksnarkException, BadItemException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -460,7 +461,6 @@ private long benchmarkVerifyOutput(String outputParams) throws ZksnarkException @Test public void calBenchmarkVerifySpend() throws ZksnarkException, BadItemException { - librustzcashInitZksnarkParams(); System.out.println("--- load ok ---"); int count = 10; @@ -492,7 +492,6 @@ public void calBenchmarkVerifySpend() throws ZksnarkException, BadItemException @Test public void calBenchmarkVerifyOutput() throws ZksnarkException, BadItemException { - librustzcashInitZksnarkParams(); System.out.println("--- load ok ---"); int count = 2; @@ -588,7 +587,6 @@ private ZenTransactionBuilder generateBuilderWithoutColumnInDescription( @Test public void testReceiveDescriptionWithEmptyCv() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -634,7 +632,6 @@ public void testReceiveDescriptionWithEmptyCv() @Test public void testReceiveDescriptionWithEmptyCm() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -677,7 +674,6 @@ public void testReceiveDescriptionWithEmptyCm() @Test public void testReceiveDescriptionWithEmptyEpk() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -719,7 +715,6 @@ public void testReceiveDescriptionWithEmptyEpk() @Test public void testReceiveDescriptionWithEmptyZkproof() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -762,7 +757,6 @@ public void testReceiveDescriptionWithEmptyZkproof() @Test public void testReceiveDescriptionWithEmptyCenc() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); dbManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); dbManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -805,7 +799,6 @@ public void testReceiveDescriptionWithEmptyCenc() @Test public void testReceiveDescriptionWithEmptyCout() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1063,7 +1056,6 @@ private ZenTransactionBuilder generateBuilder(ZenTransactionBuilder builder, lon @Test public void testReceiveDescriptionWithWrongCv() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1092,7 +1084,6 @@ public void testReceiveDescriptionWithWrongCv() @Test public void testReceiveDescriptionWithWrongZkproof() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1121,7 +1112,6 @@ public void testReceiveDescriptionWithWrongZkproof() @Test public void testReceiveDescriptionWithWrongD() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1152,7 +1142,6 @@ public void testReceiveDescriptionWithWrongD() @Test public void testReceiveDescriptionWithWrongPkd() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1183,7 +1172,6 @@ public void testReceiveDescriptionWithWrongPkd() @Test public void testReceiveDescriptionWithWrongValue() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1212,7 +1200,6 @@ public void testReceiveDescriptionWithWrongValue() @Test public void testReceiveDescriptionWithWrongRcm() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1241,7 +1228,6 @@ public void testReceiveDescriptionWithWrongRcm() @Test public void testNotMatchAskAndNsk() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1292,7 +1278,6 @@ public void testNotMatchAskAndNsk() @Test public void testRandomOvk() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1336,7 +1321,6 @@ public void testRandomOvk() //@Test not used public void testSameInputCm() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1385,7 +1369,6 @@ public void testSameInputCm() @Test public void testSameOutputCm() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1437,7 +1420,6 @@ public void testSameOutputCm() @Test public void testShieldInputInsufficient() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1482,7 +1464,6 @@ public void testShieldInputInsufficient() */ @Test public void testTransparentInputInsufficient() throws RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); long ctx = JLibrustzcash.librustzcashSaplingProvingCtxInit(); @@ -1731,7 +1712,6 @@ public TransactionCapsule generateTransactionCapsule(ZenTransactionBuilder build public void testSignWithoutFromAddress() throws BadItemException, ContractValidateException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1771,7 +1751,6 @@ public void testSignWithoutFromAddress() public void testSignWithoutFromAmout() throws BadItemException, ContractValidateException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1810,7 +1789,6 @@ public void testSignWithoutFromAmout() @Test public void testSignWithoutSpendDescription() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1856,7 +1834,6 @@ public void testSignWithoutSpendDescription() @Test public void testSignWithoutReceiveDescription() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1903,7 +1880,6 @@ public void testSignWithoutReceiveDescription() public void testSignWithoutToAddress() throws BadItemException, ContractValidateException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1943,7 +1919,6 @@ public void testSignWithoutToAddress() public void testSignWithoutToAmount() throws BadItemException, ContractValidateException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -1982,7 +1957,6 @@ public void testSignWithoutToAmount() @Test public void testSpendSignatureWithWrongColumn() throws BadItemException, RuntimeException, ZksnarkException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -2100,7 +2074,6 @@ public void testSpendSignatureWithWrongColumn() @Test public void testIsolateSignature() throws ZksnarkException, BadItemException, ContractValidateException, ContractExeException { - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -2235,7 +2208,6 @@ public void testMemoTooLong() throws ContractValidateException, TooBigTransactio AccountResourceInsufficientException, InvalidProtocolBufferException, ZksnarkException { long ctx = JLibrustzcash.librustzcashSaplingProvingCtxInit(); - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -2318,7 +2290,6 @@ public void testMemoNotEnough() throws ContractValidateException, TooBigTransact AccountResourceInsufficientException, InvalidProtocolBufferException, ZksnarkException { long ctx = JLibrustzcash.librustzcashSaplingProvingCtxInit(); - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(100 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); @@ -2412,7 +2383,6 @@ public void pushSameSkAndScanAndSpend() throws Exception { dbManager.pushBlock(new BlockCapsule(block)); //create transactions - librustzcashInitZksnarkParams(); chainBaseManager.getDynamicPropertiesStore().saveAllowShieldedTransaction(1); chainBaseManager.getDynamicPropertiesStore().saveTotalShieldedPoolValue(1000 * 1000000L); ZenTransactionBuilder builder = new ZenTransactionBuilder(wallet); diff --git a/framework/src/test/java/org/tron/keystroe/CredentialsTest.java b/framework/src/test/java/org/tron/keystroe/CredentialsTest.java index ce992c3443f..2642129e00a 100644 --- a/framework/src/test/java/org/tron/keystroe/CredentialsTest.java +++ b/framework/src/test/java/org/tron/keystroe/CredentialsTest.java @@ -1,6 +1,5 @@ package org.tron.keystroe; -import lombok.var; import org.junit.Assert; import org.junit.Test; import org.mockito.Mockito; @@ -11,24 +10,24 @@ public class CredentialsTest { @Test public void test_equality() { - var aObject = new Object(); - var si = Mockito.mock(SignInterface.class); - var si2 = Mockito.mock(SignInterface.class); - var si3 = Mockito.mock(SignInterface.class); - var address = "TQhZ7W1RudxFdzJMw6FvMnujPxrS6sFfmj".getBytes(); - var address2 = "TNCmcTdyrYKMtmE1KU2itzeCX76jGm5Not".getBytes(); + Object aObject = new Object(); + SignInterface si = Mockito.mock(SignInterface.class); + SignInterface si2 = Mockito.mock(SignInterface.class); + SignInterface si3 = Mockito.mock(SignInterface.class); + byte[] address = "TQhZ7W1RudxFdzJMw6FvMnujPxrS6sFfmj".getBytes(); + byte[] address2 = "TNCmcTdyrYKMtmE1KU2itzeCX76jGm5Not".getBytes(); Mockito.when(si.getAddress()).thenReturn(address); Mockito.when(si2.getAddress()).thenReturn(address); Mockito.when(si3.getAddress()).thenReturn(address2); - var aCredential = Credentials.create(si); + Credentials aCredential = Credentials.create(si); Assert.assertFalse(aObject.equals(aCredential)); Assert.assertFalse(aCredential.equals(aObject)); Assert.assertFalse(aCredential.equals(null)); - var anotherCredential = Credentials.create(si); - Assert.assertTrue(aCredential.equals(anotherCredential)); - var aCredential2 = Credentials.create(si2); + Credentials anotherCredential = Credentials.create(si); Assert.assertTrue(aCredential.equals(anotherCredential)); - var aCredential3 = Credentials.create(si3); + Credentials aCredential2 = Credentials.create(si2); + Assert.assertTrue(aCredential.equals(anotherCredential)); + Credentials aCredential3 = Credentials.create(si3); Assert.assertFalse(aCredential.equals(aCredential3)); } } diff --git a/framework/src/test/java/org/tron/program/DBConvertTest.java b/framework/src/test/java/org/tron/program/DBConvertTest.java deleted file mode 100644 index 7b3f797d627..00000000000 --- a/framework/src/test/java/org/tron/program/DBConvertTest.java +++ /dev/null @@ -1,116 +0,0 @@ -package org.tron.program; - -import static org.fusesource.leveldbjni.JniDBFactory.factory; - -import java.io.File; -import java.io.IOException; -import java.nio.charset.StandardCharsets; -import java.util.UUID; -import org.iq80.leveldb.DB; -import org.iq80.leveldb.Options; -import org.junit.AfterClass; -import org.junit.Assert; -import org.junit.BeforeClass; -import org.junit.Test; -import org.tron.common.utils.ByteArray; -import org.tron.common.utils.FileUtil; -import org.tron.common.utils.MarketOrderPriceComparatorForLevelDB; -import org.tron.core.capsule.MarketOrderIdListCapsule; -import org.tron.core.capsule.utils.MarketUtils; - -public class DBConvertTest { - - - private static final String INPUT_DIRECTORY = "output-directory/convert-database/"; - private static final String OUTPUT_DIRECTORY = "output-directory/convert-database-dest/"; - private static final String ACCOUNT = "account"; - private static final String MARKET = "market_pair_price_to_order"; - - - @BeforeClass - public static void init() throws IOException { - if (new File(INPUT_DIRECTORY).mkdirs()) { - initDB(new File(INPUT_DIRECTORY,ACCOUNT)); - initDB(new File(INPUT_DIRECTORY,MARKET)); - } - } - - private static void initDB(File file) throws IOException { - Options dbOptions = DBConvert.newDefaultLevelDbOptions(); - if (file.getName().contains("market_pair_price_to_order")) { - dbOptions.comparator(new MarketOrderPriceComparatorForLevelDB()); - try (DB db = factory.open(file,dbOptions)) { - - byte[] sellTokenID1 = ByteArray.fromString("100"); - byte[] buyTokenID1 = ByteArray.fromString("200"); - byte[] pairPriceKey1 = MarketUtils.createPairPriceKey( - sellTokenID1, - buyTokenID1, - 1000L, - 2001L - ); - byte[] pairPriceKey2 = MarketUtils.createPairPriceKey( - sellTokenID1, - buyTokenID1, - 1000L, - 2002L - ); - byte[] pairPriceKey3 = MarketUtils.createPairPriceKey( - sellTokenID1, - buyTokenID1, - 1000L, - 2003L - ); - - MarketOrderIdListCapsule capsule1 = new MarketOrderIdListCapsule(ByteArray.fromLong(1), - ByteArray.fromLong(1)); - MarketOrderIdListCapsule capsule2 = new MarketOrderIdListCapsule(ByteArray.fromLong(2), - ByteArray.fromLong(2)); - MarketOrderIdListCapsule capsule3 = new MarketOrderIdListCapsule(ByteArray.fromLong(3), - ByteArray.fromLong(3)); - - //Use out-of-order insertion,key in store should be 1,2,3 - db.put(pairPriceKey1, capsule1.getData()); - db.put(pairPriceKey2, capsule2.getData()); - db.put(pairPriceKey3, capsule3.getData()); - } - - } else { - try (DB db = factory.open(file,dbOptions)) { - for (int i = 0; i < 100; i++) { - byte[] bytes = UUID.randomUUID().toString().getBytes(StandardCharsets.UTF_8); - db.put(bytes, bytes); - } - } - } - - } - - @AfterClass - public static void destroy() { - FileUtil.deleteDir(new File(INPUT_DIRECTORY)); - FileUtil.deleteDir(new File(OUTPUT_DIRECTORY)); - } - - @Test - public void testRun() { - String[] args = new String[] { INPUT_DIRECTORY, OUTPUT_DIRECTORY }; - Assert.assertEquals(0, DBConvert.run(args)); - } - - @Test - public void testNotExist() { - String[] args = new String[] {OUTPUT_DIRECTORY + File.separator + UUID.randomUUID(), - OUTPUT_DIRECTORY}; - Assert.assertEquals(404, DBConvert.run(args)); - } - - @Test - public void testEmpty() { - File file = new File(OUTPUT_DIRECTORY + File.separator + UUID.randomUUID()); - file.mkdirs(); - file.deleteOnExit(); - String[] args = new String[] {file.toString(), OUTPUT_DIRECTORY}; - Assert.assertEquals(0, DBConvert.run(args)); - } -} diff --git a/framework/src/test/java/org/tron/program/SolidityNodeTest.java b/framework/src/test/java/org/tron/program/SolidityNodeTest.java index a95d07c0c11..943c73cb9ed 100755 --- a/framework/src/test/java/org/tron/program/SolidityNodeTest.java +++ b/framework/src/test/java/org/tron/program/SolidityNodeTest.java @@ -1,13 +1,21 @@ package org.tron.program; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertThrows; + +import java.util.concurrent.TimeUnit; import javax.annotation.Resource; import lombok.extern.slf4j.Slf4j; import org.junit.Assert; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.Timeout; import org.tron.common.BaseTest; import org.tron.common.client.DatabaseGrpcClient; +import org.tron.common.utils.PublicMethod; import org.tron.core.Constant; import org.tron.core.config.args.Args; +import org.tron.core.exception.TronError; import org.tron.core.services.RpcApiService; import org.tron.core.services.http.solidity.SolidityNodeHttpApiService; import org.tron.protos.Protocol.Block; @@ -20,23 +28,35 @@ public class SolidityNodeTest extends BaseTest { RpcApiService rpcApiService; @Resource SolidityNodeHttpApiService solidityNodeHttpApiService; + static int rpcPort = PublicMethod.chooseRandomPort(); + static int solidityHttpPort = PublicMethod.chooseRandomPort(); + + @Rule + public Timeout timeout = new Timeout(30, TimeUnit.SECONDS); static { - Args.setParam(new String[]{"-d", dbPath()}, Constant.TEST_CONF); - Args.getInstance().setSolidityNode(true); + Args.setParam(new String[] {"-d", dbPath(), "--solidity"}, Constant.TEST_CONF); + Args.getInstance().setRpcPort(rpcPort); + Args.getInstance().setSolidityHttpPort(solidityHttpPort); } @Test public void testSolidityArgs() { Assert.assertNotNull(Args.getInstance().getTrustNodeAddr()); Assert.assertTrue(Args.getInstance().isSolidityNode()); + String trustNodeAddr = Args.getInstance().getTrustNodeAddr(); + Args.getInstance().setTrustNodeAddr(null); + TronError thrown = assertThrows(TronError.class, + SolidityNode::start); + assertEquals(TronError.ErrCode.SOLID_NODE_INIT, thrown.getErrCode()); + Args.getInstance().setTrustNodeAddr(trustNodeAddr); } @Test public void testSolidityGrpcCall() { rpcApiService.start(); DatabaseGrpcClient databaseGrpcClient = null; - String address = Args.getInstance().getTrustNodeAddr(); + String address = Args.getInstance().getTrustNodeAddr().split(":")[0] + ":" + rpcPort; try { databaseGrpcClient = new DatabaseGrpcClient(address); } catch (Exception e) { diff --git a/framework/src/test/resources/config-localtest.conf b/framework/src/test/resources/config-localtest.conf index ff31369a915..8049ceb6cda 100644 --- a/framework/src/test/resources/config-localtest.conf +++ b/framework/src/test/resources/config-localtest.conf @@ -90,7 +90,7 @@ node { minParticipationRate = 0 - fullNodeAllowShieldedTransaction = true + allowShieldedTransactionApi = true zenTokenId = 1000001 @@ -167,6 +167,7 @@ node { # httpPBFTPort = 8565 # maxBlockRange = 5000 # maxSubTopics = 1000 + # maxBlockFilterNum = 30000 } } diff --git a/framework/src/test/resources/config-test-index.conf b/framework/src/test/resources/config-test-index.conf index faa2f93dc5e..3ea6b50b20c 100644 --- a/framework/src/test/resources/config-test-index.conf +++ b/framework/src/test/resources/config-test-index.conf @@ -88,6 +88,7 @@ node { httpPBFTEnable = false # maxBlockRange = 5000 # maxSubTopics = 1000 + # maxBlockFilterNum = 30000 } rpc { diff --git a/framework/src/test/resources/config-test-mainnet.conf b/framework/src/test/resources/config-test-mainnet.conf index 12acad64d8d..123c8e5d368 100644 --- a/framework/src/test/resources/config-test-mainnet.conf +++ b/framework/src/test/resources/config-test-mainnet.conf @@ -94,6 +94,7 @@ node { httpPBFTEnable = false # maxBlockRange = 5000 # maxSubTopics = 1000 + # maxBlockFilterNum = 50000 } rpc { diff --git a/framework/src/test/resources/config-test.conf b/framework/src/test/resources/config-test.conf index eaa6659a8c4..eb4f605ab91 100644 --- a/framework/src/test/resources/config-test.conf +++ b/framework/src/test/resources/config-test.conf @@ -118,6 +118,7 @@ node { httpPBFTEnable = false # maxBlockRange = 5000 # maxSubTopics = 1000 + # maxBlockFilterNum = 30000 } # use your ipv6 address for node discovery and tcp connection, default false @@ -209,6 +210,8 @@ node { # The switch of the reflection service, effective for all gRPC services reflectionService = true + maxRstStream = 5 + secondsPerWindow = 10 } } @@ -327,7 +330,7 @@ genesis.block = { voteCount = 96 }, { - address: 27VZHn9PFZwNh7o2EporxmLkpe157iWZVkh + address: 27bi7CD8d94AgXY3XFS9A9vx78Si5MqrECz url = "http://AlphaLyrae.org", voteCount = 95 } diff --git a/framework/src/test/resources/logback-test.xml b/framework/src/test/resources/logback-test.xml index d05c7fca79e..9cf4a04062f 100644 --- a/framework/src/test/resources/logback-test.xml +++ b/framework/src/test/resources/logback-test.xml @@ -34,7 +34,7 @@ - - + + diff --git a/gradle/jdk17/java-tron.vmoptions b/gradle/jdk17/java-tron.vmoptions new file mode 100644 index 00000000000..7af3123d268 --- /dev/null +++ b/gradle/jdk17/java-tron.vmoptions @@ -0,0 +1,8 @@ +-XX:+UseZGC +-Xlog:gc,gc+heap:file=gc.log:time,tags,level:filecount=10,filesize=100M +-XX:ReservedCodeCacheSize=256m +-XX:+UseCodeCacheFlushing +-XX:MetaspaceSize=256m +-XX:MaxMetaspaceSize=512m +-XX:MaxDirectMemorySize=1g +-XX:+HeapDumpOnOutOfMemoryError \ No newline at end of file diff --git a/gradle/verification-metadata.xml b/gradle/verification-metadata.xml index b3c879f6b40..4d0bf1013d6 100644 --- a/gradle/verification-metadata.xml +++ b/gradle/verification-metadata.xml @@ -6,6 +6,7 @@ + @@ -150,28 +151,12 @@ - - - + + + - - - - - - - - - - - - - - - - - - + + @@ -179,9 +164,9 @@ - - - + + + @@ -189,9 +174,9 @@ - - - + + + @@ -199,9 +184,9 @@ - - - + + + @@ -209,9 +194,9 @@ - - - + + + @@ -219,15 +204,15 @@ - - - + + + - - + + - - + + @@ -235,15 +220,15 @@ - - - + + + - - + + - - + + @@ -251,15 +236,15 @@ - - - + + + - - + + - - + + @@ -269,6 +254,9 @@ + + + @@ -304,12 +292,12 @@ - - - + + + - - + + @@ -336,12 +324,12 @@ - - - + + + - - + + @@ -352,9 +340,9 @@ - - - + + + @@ -375,12 +363,12 @@ - - - + + + - - + + @@ -393,9 +381,9 @@ - - - + + + @@ -406,12 +394,12 @@ - - - + + + - - + + @@ -445,6 +433,28 @@ + + + + + + + + + + + + + + + + + + + + + + @@ -475,6 +485,16 @@ + + + + + + + + + + @@ -483,19 +503,6 @@ - - - - - - - - - - - - - @@ -505,16 +512,21 @@ - - - - - - + + + + + + + + + + + @@ -525,36 +537,42 @@ - - - + + + - - + + - - - + + + - - + + - - - + + + - - - + + + + + + - - + + - - + + + + + @@ -728,6 +746,14 @@ + + + + + + + + @@ -820,12 +846,15 @@ - - - + + + + + + - - + + @@ -836,82 +865,88 @@ - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + + + + + + + @@ -919,105 +954,116 @@ - - - + + + + + + + + + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + + + + - - + + @@ -1086,25 +1132,12 @@ - - - + + + - - - - - - - - - - - - - - - + + @@ -1139,6 +1172,14 @@ + + + + + + + + @@ -1298,9 +1339,14 @@ - - - + + + + + + + + @@ -1337,6 +1383,14 @@ + + + + + + + + @@ -1393,16 +1447,21 @@ - - - - - + + + + + + + + + + @@ -1540,28 +1599,28 @@ - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + @@ -1572,12 +1631,12 @@ - - - + + + - - + + @@ -1593,12 +1652,15 @@ - - - + + + + + + - - + + @@ -1614,12 +1676,12 @@ - - - + + + - - + + @@ -1627,9 +1689,9 @@ - - - + + + @@ -1637,9 +1699,9 @@ - - - + + + @@ -1666,65 +1728,65 @@ - - - + + + - - + + - - - + + + - - + + - - - + + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + @@ -1807,14 +1869,6 @@ - - - - - - - - @@ -1823,27 +1877,19 @@ - - - - - - - - - - - + + + - - - + + + @@ -1943,6 +1989,38 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -2080,12 +2158,12 @@ - - - + + + - - + + @@ -2112,6 +2190,14 @@ + + + + + + + + @@ -2207,81 +2293,86 @@ - - - - - - + + + - - - - + + - - + + - - - - - - + + + - - - - + + - - + + - - - + + + - - + + + + + - - - + + + + + + - - + + - - - + + + + + + - - + + - - - + + + - - + + + + + - - - + + + + + + - - + + - - - + + + diff --git a/install_dependencies.sh b/install_dependencies.sh new file mode 100755 index 00000000000..f72ecf2e192 --- /dev/null +++ b/install_dependencies.sh @@ -0,0 +1,957 @@ +#!/usr/bin/env bash + +set -e + +OS="$(uname -s)" +ARCH="$(uname -m)" + +echo "========================================" +echo " TRON Java Dependencies Installer" +echo "========================================" +echo "" +echo ">>> Environment Detection" +if [[ "$OS" == "Darwin" ]]; then + echo " OS: macOS $OS" +else + echo " OS: $OS" +fi +echo " Architecture: $ARCH" + +# Validate OS and architecture support first +if [[ "$OS" != "Darwin" && "$OS" != "Linux" ]]; then + echo "Error: Unsupported OS $OS" + exit 1 +elif [[ "$OS" == "Darwin" ]]; then + if [[ "$ARCH" != "x86_64" && "$ARCH" != "arm64" ]]; then + echo "Error: Unsupported architecture for macOS: $ARCH" + exit 1 + fi +else + if [[ "$ARCH" != "x86_64" && "$ARCH" != "aarch64" && "$ARCH" != "arm64" ]]; then + echo "Error: Unsupported architecture for Linux: $ARCH" + exit 1 + fi +fi + +echo "" +echo ">>> Tested platforms:" +echo " - macOS x86_64 (JDK 8)" +echo " - macOS arm64 (JDK 17)" +echo " - Linux x86_64 (generic, including Ubuntu) (JDK 8)" +echo " - Linux arm64/aarch64 (generic, including Ubuntu) (JDK 17)" +echo " Note: Other platforms may require manual installation if errors occur" +echo "" +echo ">>> This script will install the following components if not already installed:" +if [[ "$OS" == "Darwin" ]]; then + echo " 1. Homebrew to download and install JDK (macOS only)" + echo " 2. Git for cloning Github repository" +else + echo " 1. Git for cloning Github repository" +fi +if [[ "$OS" == "Darwin" ]]; then + if [[ "$ARCH" == "x86_64" ]]; then + echo " 3. OpenJDK 8 (required for x86_64 architecture)" + else + echo " 3. OpenJDK 17 (required for arm64 architecture)" + fi +else + if [[ "$ARCH" == "x86_64" ]]; then + echo " 2. OpenJDK 8 (required for x86_64 architecture)" + else + echo " 2. OpenJDK 17 (required for arm64/aarch64 architecture)" + fi +fi +echo "" + +# Function to ask for user confirmation +ask_confirmation() { + while true; do + read -p "Do you want to continue? (y/N): " yn + case $yn in + [Yy]* ) return 0;; + [Nn]* | "" ) echo "Installation cancelled."; exit 0;; + * ) echo "Please answer yes (y) or no (n).";; + esac + done +} + +# Function to check Java version +check_java_version() { + if command -v java &> /dev/null; then + local java_version=$(java -version 2>&1 | head -n 1 | cut -d'"' -f2) + echo " Current Java version: $java_version" + + # Check if it's JDK 8 (version starts with 1.8) + if [[ "$java_version" =~ ^1\.8\. ]]; then + echo " JDK 8 is installed." + return 0 + # Check if it's JDK 17 (version starts with 17) + elif [[ "$java_version" =~ ^17\. ]]; then + echo " JDK 17 is installed." + return 1 + else + echo " Different Java version detected: $java_version" + return 2 + fi + else + return 3 + fi +} + +# Function to ask for JDK installation confirmation +ask_jdk_confirmation() { + local current_version="$1" + local required_version="$2" + local arch="$3" + + echo "" + echo "JDK Version Mismatch Detected!" + echo " Current version: $current_version" + echo " Current installation path: $(which java 2>/dev/null || echo 'Not found')" + if command -v java &> /dev/null && [[ -n "$JAVA_HOME" ]]; then + echo " Current JAVA_HOME: $JAVA_HOME" + fi + echo " Required version for $arch: $required_version" + echo " This script will install $required_version alongside your existing installation." + echo " Your current Java installation will not be removed." + echo "" + + while true; do + read -p "Do you want to install $required_version? (y/N): " yn + case $yn in + [Yy]* ) return 0;; + [Nn]* | "" ) echo "JDK installation cancelled. Exiting."; exit 0;; + * ) echo "Please answer yes (y) or no (n).";; + esac + done +} + +# First, check and install Git (needed for cloning repository) +echo ">>> Checking Git installation..." +if ! command -v git &> /dev/null; then + echo " Git is not installed." + while true; do + read -p "Do you want to install Git (required for cloning the java-tron repository)? (y/N): " yn + case $yn in + [Yy]* ) + echo ">>> Installing Git..." + INSTALL_GIT=true + break;; + [Nn]* | "" ) + echo "Git installation cancelled. You'll need Git to clone the java-tron repository." + echo "You can install Git manually later and then clone the repository." + INSTALL_GIT=false + break;; + * ) echo "Please answer yes (y) or no (n).";; + esac + done +else + echo "Git is already installed: $(git --version)" + INSTALL_GIT=false +fi + +echo "" +echo ">>> Checking existing Java installation..." +set +e # Temporarily disable exit on error +check_java_version +java_status=$? +set -e # Re-enable exit on error + +# Determine required JDK version based on architecture +if [[ "$OS" == "Darwin" ]]; then + if [[ "$ARCH" == "x86_64" ]]; then + required_jdk="JDK 8" + required_status=0 + elif [[ "$ARCH" == "arm64" ]]; then + required_jdk="JDK 17" + required_status=1 + fi +elif [[ "$OS" == "Linux" ]]; then + if [[ "$ARCH" == "x86_64" ]]; then + required_jdk="JDK 8" + required_status=0 + elif [[ "$ARCH" == "aarch64" ]] || [[ "$ARCH" == "arm64" ]]; then + required_jdk="JDK 17" + required_status=1 + fi +fi + +# Check if correct JDK version is already installed +if [[ $java_status -eq $required_status ]]; then + echo " You can skip the Java installation part." + echo "" + if [[ "$INSTALL_GIT" == "false" ]]; then + echo "Both Git and Java JDK are ready for TRON development!" + echo "" + exit 0 + else + echo ">>> Proceeding with Git installation only..." + SKIP_JAVA_INSTALL=true + fi +elif [[ $java_status -eq 0 ]] || [[ $java_status -eq 1 ]] || [[ $java_status -eq 2 ]]; then + # Different JDK version is installed, ask for confirmation + current_version=$(java -version 2>&1 | head -n 1 | cut -d'"' -f2) + ask_jdk_confirmation "$current_version" "$required_jdk" "$ARCH" + SKIP_JAVA_INSTALL=false +else + # No Java installation found, ask for general confirmation + echo "" + echo "No Java installation detected!" + echo " This script will install $required_jdk which is required for $ARCH architecture." + echo "" + ask_confirmation + SKIP_JAVA_INSTALL=false +fi + +# Function to show permanent Java configuration instructions +show_permanent_java_config() { + local jdk_version="$1" + local os_type="$2" + local java_home="$3" + local java_bin_path="$4" + + echo "" + echo " To make JDK $jdk_version permanent:" + if [[ "$os_type" == "Darwin" ]]; then + echo " # Add to ~/.zshrc or ~/.bash_profile:" + echo " echo 'export JAVA_HOME=\"$java_home\"' >> ~/.zshrc" + echo " echo 'export PATH=\"\$JAVA_HOME/bin:\$PATH\"' >> ~/.zshrc" + echo " # Then run below command:" + echo " source ~/.zshrc" + echo "" + echo " # Or use jenv for Java version management:" + echo " brew install jenv" + echo " jenv add $java_home" + elif [[ "$os_type" == "Linux" ]]; then + echo " # Method 1: Add to ~/.bashrc:" + echo " echo 'export JAVA_HOME=\"$java_home\"' >> ~/.bashrc" + echo " echo 'export PATH=\"\$JAVA_HOME/bin:\$PATH\"' >> ~/.bashrc" + echo " source ~/.bashrc" + echo "" + echo " # Method 2: Use update-alternatives (recommended):" + if [[ "$PKG_MANAGER" == "apt-get" ]]; then + echo " sudo update-alternatives --install /usr/bin/java java $java_bin_path/java 1" + echo " sudo update-alternatives --install /usr/bin/javac javac $java_bin_path/javac 1" + echo " sudo update-alternatives --config java" + else + echo " sudo alternatives --install /usr/bin/java java $java_bin_path/java 1" + echo " sudo alternatives --install /usr/bin/javac javac $java_bin_path/javac 1" + echo " sudo alternatives --config java" + fi + fi + echo "" +} + +# Function to show Java environment application instructions +show_java_env_instructions() { + local java_home="$1" + local java_bin_path="$2" + + echo "" + echo " ✓ Java environment has been applied to this script session." + echo " To apply Java environment to your current terminal session:" + echo " source ./tron_java_env.sh" + echo "" + echo " Or run these commands directly:" + echo " export JAVA_HOME=\"$java_home\"" + echo " export PATH=\"$java_bin_path:\$PATH\"" + echo "" + echo " Note: You may need to open a new terminal or run 'source ./tron_java_env.sh'" + echo " if 'java -version' doesn't work immediately after this script completes." +} + +# Function to get Java paths based on OS and architecture +get_java_paths() { + local jdk_version="$1" + local os_type="$2" + local arch="$3" + local java_home="" + + if [[ "$os_type" == "Darwin" ]]; then + # macOS paths - try to detect actual Homebrew installation + if [[ "$jdk_version" == "8" ]]; then + # Try multiple possible paths for JDK 8 + for path in "/usr/local/opt/openjdk@8" "/opt/homebrew/opt/openjdk@8"; do + if [[ -d "$path" ]]; then + java_home="$path" + break + fi + done + # If not found, use brew --prefix to get the correct path + if [[ -z "$java_home" ]] && command -v brew &> /dev/null; then + local brew_prefix=$(brew --prefix openjdk@8 2>/dev/null || echo "") + if [[ -n "$brew_prefix" && -d "$brew_prefix" ]]; then + java_home="$brew_prefix" + fi + fi + elif [[ "$jdk_version" == "17" ]]; then + # Try multiple possible paths for JDK 17 + for path in "/opt/homebrew/opt/openjdk@17" "/usr/local/opt/openjdk@17"; do + if [[ -d "$path" ]]; then + java_home="$path" + break + fi + done + # If not found, use brew --prefix to get the correct path + if [[ -z "$java_home" ]] && command -v brew &> /dev/null; then + local brew_prefix=$(brew --prefix openjdk@17 2>/dev/null || echo "") + if [[ -n "$brew_prefix" && -d "$brew_prefix" ]]; then + java_home="$brew_prefix" + fi + fi + fi + elif [[ "$os_type" == "Linux" ]]; then + # Linux paths - provide generic path for manual configuration + java_home="/usr/lib/jvm/java-$jdk_version-openjdk" + fi + + echo "$java_home" +} + +# Unified Java environment configuration function +configure_java_environment() { + local jdk_version="$1" + local os_type="$2" + local arch="$3" + local java_home="" + local java_bin_path="" + + echo "" + echo ">>> Configuring Java environment for JDK $jdk_version..." + + # Ask user for confirmation before changing environment + echo "" + echo "This will modify your Java environment settings:" + echo " - Set JAVA_HOME to the new JDK $jdk_version installation" + echo " - Update PATH to include the new Java binaries" + echo " - Create a script (tron_java_env.sh) for easy environment setup" + echo "" + + while true; do + read -p "Do you want to configure the Java environment for JDK $jdk_version? (y/N): " yn + case $yn in + [Yy]* ) + echo ">>> Proceeding with Java environment configuration..." + break;; + [Nn]* | "" ) + echo "Java environment configuration skipped." + echo "You may need to manually set JAVA_HOME and PATH for JDK $jdk_version" + echo "" + echo "Manual configuration commands:" + + # Get the expected Java path + local expected_java_home=$(get_java_paths "$jdk_version" "$os_type" "$arch") + local expected_java_bin_path="$expected_java_home/bin" + echo " export JAVA_HOME=\"$expected_java_home\"" + echo " export PATH=\"\$JAVA_HOME/bin:\$PATH\"" + + if [[ "$os_type" == "Linux" ]]; then + echo "" + echo "Note: Actual path may vary depending on your distribution." + echo "Common paths include:" + echo " /usr/lib/jvm/java-$jdk_version-openjdk-amd64 (Ubuntu/Debian)" + echo " /usr/lib/jvm/java-1.$jdk_version.0-openjdk (RHEL/CentOS)" + fi + + # Create tron_java_env.sh even when user skips configuration + echo "" + echo ">>> Creating tron_java_env.sh for manual use later..." + local env_script="./tron_java_env.sh" + cat > "$env_script" << EOF +#!/bin/bash +# TRON Java Environment Configuration +# Generated by install_dependencies.sh on $(date) + +export JAVA_HOME="$expected_java_home" +export PATH="$expected_java_bin_path:\$PATH" + +echo "Java environment configured:" +echo " JAVA_HOME: \$JAVA_HOME" +echo " Java version: \$(java -version 2>&1 | head -n 1)" +EOF + chmod +x "$env_script" + echo " ✓ Created $env_script" + + # Show the same application instructions as automatic configuration + show_java_env_instructions "$expected_java_home" "$expected_java_bin_path" + + # Show permanent configuration instructions + show_permanent_java_config "$jdk_version" "$os_type" "$expected_java_home" "$expected_java_bin_path" + + echo "" + return 1;; + * ) echo "Please answer yes (y) or no (n).";; + esac + done + + # Determine Java paths based on OS and architecture + if [[ "$os_type" == "Darwin" ]]; then + # Use the helper function for macOS + java_home=$(get_java_paths "$jdk_version" "$os_type" "$arch") + java_bin_path="$java_home/bin" + + # Debug output for macOS + echo " Detected Java path: $java_home" + if [[ ! -d "$java_home" ]]; then + echo " Warning: Java home directory not found at: $java_home" + echo " Attempting to find JDK installation..." + + # Try to find the installation using brew + if command -v brew &> /dev/null; then + local brew_list=$(brew list --formula | grep "openjdk@$jdk_version" || echo "") + if [[ -n "$brew_list" ]]; then + echo " Found Homebrew package: $brew_list" + local brew_prefix=$(brew --prefix openjdk@$jdk_version 2>/dev/null || echo "") + if [[ -n "$brew_prefix" && -d "$brew_prefix" ]]; then + java_home="$brew_prefix" + java_bin_path="$java_home/bin" + echo " Updated Java path to: $java_home" + fi + else + echo " Error: openjdk@$jdk_version not found in Homebrew packages" + echo " Try running: brew list | grep openjdk" + return 1 + fi + fi + fi + elif [[ "$os_type" == "Linux" ]]; then + # Linux paths - try to find the actual installation + if [[ "$jdk_version" == "8" ]]; then + if [[ "$PKG_MANAGER" == "apt-get" ]]; then + if [[ "$arch" == "aarch64" ]] || [[ "$arch" == "arm64" ]]; then + java_home="/usr/lib/jvm/java-8-openjdk-arm64" + else + java_home="/usr/lib/jvm/java-8-openjdk-amd64" + fi + else + # RHEL/CentOS/Amazon Linux - try multiple possible paths + for path in "/usr/lib/jvm/java-1.8.0-amazon-corretto" "/usr/lib/jvm/java-1.8.0-openjdk"; do + if [[ -d "$path" ]]; then + java_home="$path" + break + fi + done + fi + elif [[ "$jdk_version" == "17" ]]; then + if [[ "$PKG_MANAGER" == "apt-get" ]]; then + if [[ "$arch" == "aarch64" ]] || [[ "$arch" == "arm64" ]]; then + java_home="/usr/lib/jvm/java-17-openjdk-arm64" + else + java_home="/usr/lib/jvm/java-17-openjdk-amd64" + fi + else + # RHEL/CentOS/Amazon Linux - try multiple possible paths + for path in "/usr/lib/jvm/java-17-amazon-corretto" "/usr/lib/jvm/java-17-openjdk"; do + if [[ -d "$path" ]]; then + java_home="$path" + break + fi + done + fi + fi + java_bin_path="$java_home/bin" + fi + + # Set environment variables for current session + if [[ -d "$java_home" ]]; then + export JAVA_HOME="$java_home" + export PATH="$java_bin_path:$PATH" + echo " JAVA_HOME set to: $JAVA_HOME" + echo " PATH updated to include: $java_bin_path" + echo " Environment temporarily configured for JDK $jdk_version" + + # Create a source script for the user's current shell + local env_script="./tron_java_env.sh" + cat > "$env_script" << EOF +#!/bin/bash +# TRON Java Environment Configuration +# Generated by install_dependencies.sh on $(date) + +export JAVA_HOME="$java_home" +export PATH="$java_bin_path:\$PATH" + +echo "Java environment configured:" +echo " JAVA_HOME: \$JAVA_HOME" +echo " Java version: \$(java -version 2>&1 | head -n 1)" +EOF + chmod +x "$env_script" + echo "" + echo " ✓ Created $env_script" + + echo "" + echo " Applying Java environment to current shell session..." + # Source the environment script to apply it immediately + source "$env_script" + + show_java_env_instructions "$java_home" "$java_bin_path" + + else + echo " Could not find Java installation at expected path: $java_home" + echo " You may need to set JAVA_HOME manually" + return 1 + fi + + # Provide OS-specific permanent configuration instructions + show_permanent_java_config "$jdk_version" "$os_type" "$java_home" "$java_bin_path" + + return 0 +} + +echo "----------------------------------------" + +install_macos() { + if ! command -v brew &> /dev/null; then + echo ">>> Homebrew not found." + echo " Homebrew is required to install Java on macOS." + echo "" + while true; do + read -p "Do you want to install Homebrew? (y/N): " yn + case $yn in + [Yy]* ) + echo ">>> Installing Homebrew..." + /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" + break;; + [Nn]* | "" ) + echo "Homebrew installation cancelled." + echo "Cannot proceed with Java installation without Homebrew on macOS." + echo "Please install Homebrew manually or use alternative Java installation methods." + exit 1;; + * ) echo "Please answer yes (y) or no (n).";; + esac + done + + # Add Homebrew to PATH for the current session (Apple Silicon vs Intel) + if [[ "$ARCH" == "arm64" ]]; then + eval "$(/opt/homebrew/bin/brew shellenv)" + else + eval "$(/usr/local/bin/brew shellenv)" + fi + else + echo ">>> Homebrew is already installed." + fi + + echo ">>> Updating Homebrew..." + brew update + + # Install Git if needed + if [[ "$INSTALL_GIT" == "true" ]]; then + echo ">>> Installing Git..." + brew install git + echo " Git installed successfully: $(git --version)" + fi + + # Skip Java installation if flag is set + if [[ "$SKIP_JAVA_INSTALL" == "true" ]]; then + echo ">>> Skipping Java installation (correct version already detected)." + return 0 + fi + + if [[ "$ARCH" == "x86_64" ]]; then + echo ">>> Architecture is x86_64. Checking for JDK 8..." + set +e # Temporarily disable exit on error + check_java_version + local java_status=$? + set -e # Re-enable exit on error + + if [[ $java_status -eq 0 ]]; then + echo ">>> JDK 8 is already installed. Skipping installation." + else + if [[ $java_status -eq 1 ]]; then + echo ">>> Installing JDK 8 alongside existing JDK 17..." + elif [[ $java_status -eq 2 ]]; then + echo ">>> Installing JDK 8 alongside existing Java installation..." + else + echo ">>> Installing JDK 8..." + fi + if brew install openjdk@8; then + echo ">>> JDK 8 installation completed successfully." + + # Use unified Java environment configuration + if configure_java_environment "8" "Darwin" "$ARCH"; then + echo "Environment has been updated! Java 8 is now configured." + else + echo "Java 8 installed but environment not configured. You may need to set JAVA_HOME manually." + fi + else + echo "Error: Failed to install JDK 8 via Homebrew." + echo "Please try installing manually with: brew install openjdk@8" + exit 1 + fi + fi + + elif [[ "$ARCH" == "arm64" ]]; then + echo ">>> Architecture is arm64. Checking for JDK 17..." + set +e # Temporarily disable exit on error + check_java_version + local java_status=$? + set -e # Re-enable exit on error + + if [[ $java_status -eq 1 ]]; then + echo ">>> JDK 17 is already installed. Skipping installation." + else + if [[ $java_status -eq 0 ]]; then + echo ">>> Installing JDK 17 alongside existing JDK 8..." + elif [[ $java_status -eq 2 ]]; then + echo ">>> Installing JDK 17 alongside existing Java installation..." + else + echo ">>> Installing JDK 17..." + fi + if brew install openjdk@17; then + echo ">>> JDK 17 installation completed successfully." + + # Use unified Java environment configuration + if configure_java_environment "17" "Darwin" "$ARCH"; then + echo "Environment has been updated! Java 17 is now configured." + else + echo "Java 17 installed but environment not configured. You may need to set JAVA_HOME manually." + fi + else + echo "Error: Failed to install JDK 17 via Homebrew." + echo "Please try installing manually with: brew install openjdk@17" + exit 1 + fi + fi + + else + echo "Error: Unsupported architecture for macOS script: $ARCH" + exit 1 + fi +} + +install_linux() { + if command -v dnf &> /dev/null; then + PKG_MANAGER="dnf" + INSTALL_CMD="sudo dnf install -y" + UPDATE_CMD="sudo dnf check-update" + elif command -v yum &> /dev/null; then + PKG_MANAGER="yum" + INSTALL_CMD="sudo yum install -y" + UPDATE_CMD="sudo yum check-update" + elif command -v apt-get &> /dev/null; then + PKG_MANAGER="apt-get" + INSTALL_CMD="sudo apt-get install -y" + UPDATE_CMD="sudo apt-get update" + else + echo "Error: Unsupported package manager. Only apt-get (Debian/Ubuntu) and yum/dnf (RHEL/CentOS/Amazon Linux) are currently supported." + exit 1 + fi + + echo ">>> Updating package index ($PKG_MANAGER)..." + $UPDATE_CMD || true + + # Install Git if needed + if [[ "$INSTALL_GIT" == "true" ]]; then + echo ">>> Installing Git..." + $INSTALL_CMD git + echo " Git installed successfully: $(git --version)" + fi + + # Skip Java installation if flag is set + if [[ "$SKIP_JAVA_INSTALL" == "true" ]]; then + echo ">>> Skipping Java installation (correct version already detected)." + return 0 + fi + + install_first_available() { + local target_version="$1" + shift + local installed_package="" + + for pkg in "$@"; do + echo " Attempting to install: $pkg" + if $INSTALL_CMD "$pkg"; then + installed_package="$pkg" + echo " Successfully installed: $pkg" + break + else + echo " Failed to install: $pkg" + fi + done + + if [[ -n "$installed_package" ]]; then + # Verify what version was actually installed + echo " Verifying installed Java version..." + if command -v java &> /dev/null; then + local actual_version=$(java -version 2>&1 | head -n 1 | cut -d'"' -f2) + echo " Installed Java version: $actual_version" + + # Check if the installed version matches what we expected + if [[ "$target_version" == "8" ]]; then + if [[ "$actual_version" =~ ^1\.8\. ]]; then + echo " ✓ JDK 8 installed successfully" + return 0 + else + echo " ✗ Expected JDK 8 but got: $actual_version" + echo " This may happen if JDK 8 is not available in your distribution" + return 2 + fi + elif [[ "$target_version" == "17" ]]; then + if [[ "$actual_version" =~ ^17\. ]]; then + echo " ✓ JDK 17 installed successfully" + return 0 + else + echo " ✗ Expected JDK 17 but got: $actual_version" + return 2 + fi + fi + else + echo " ✗ Java command not found after installation" + return 1 + fi + else + return 1 + fi + } + + if [[ "$ARCH" == "x86_64" ]]; then + echo ">>> Architecture is x86_64. Checking for JDK 8..." + set +e # Temporarily disable exit on error + check_java_version + local java_status=$? + set -e # Re-enable exit on error + + if [[ $java_status -eq 0 ]]; then + echo ">>> JDK 8 is already installed. Skipping installation." + else + if [[ $java_status -eq 1 ]]; then + echo ">>> Installing JDK 8 alongside existing JDK 17..." + elif [[ $java_status -eq 2 ]]; then + echo ">>> Installing JDK 8 alongside existing Java installation..." + else + echo ">>> Installing JDK 8..." + fi + if [[ "$PKG_MANAGER" == "apt-get" ]]; then + if install_first_available "8" openjdk-8-jdk; then + install_result=0 + else + install_result=$? + fi + else + if install_first_available "8" java-1.8.0-amazon-corretto-devel java-1.8.0-openjdk-devel; then + install_result=0 + else + install_result=$? + fi + fi + + if [[ $install_result -eq 0 ]]; then + # Use unified Java environment configuration + if configure_java_environment "8" "Linux" "$ARCH"; then + echo "Environment has been updated! Java 8 is now configured." + else + echo "Java 8 installed but environment not configured. You may need to set JAVA_HOME manually." + fi + elif [[ $install_result -eq 2 ]]; then + # JDK 8 package is installed but default version is different + # Need to switch to JDK 8 using update-alternatives + local actual_version=$(java -version 2>&1 | head -n 1 | cut -d'"' -f2) + echo ">>> JDK 8 package is installed, but system default is: $actual_version" + echo ">>> Switching system default to JDK 8 using update-alternatives..." + + # Try to switch to JDK 8 + if [[ "$PKG_MANAGER" == "apt-get" ]]; then + local jdk8_path="/usr/lib/jvm/java-8-openjdk-amd64" + if [[ -d "$jdk8_path" ]]; then + echo " Found JDK 8 at: $jdk8_path" + # Set JDK 8 as default using update-alternatives + sudo update-alternatives --set java "$jdk8_path/jre/bin/java" 2>/dev/null || \ + sudo update-alternatives --set java "$jdk8_path/bin/java" 2>/dev/null || \ + echo " Note: Could not auto-switch. Please run: sudo update-alternatives --config java" + + sudo update-alternatives --set javac "$jdk8_path/bin/javac" 2>/dev/null || \ + echo " Note: Could not auto-switch javac. Please run: sudo update-alternatives --config javac" + + # Verify the switch + local new_version=$(java -version 2>&1 | head -n 1 | cut -d'"' -f2) + if [[ "$new_version" =~ ^1\.8\. ]]; then + echo " ✓ Successfully switched to JDK 8: $new_version" + # Now configure environment for JDK 8 + if configure_java_environment "8" "Linux" "$ARCH"; then + echo "Environment has been updated! Java 8 is now configured." + else + echo "Java 8 is active but environment not configured. You may need to set JAVA_HOME manually." + fi + else + echo " ✗ Auto-switch failed. Current version: $new_version" + echo " Please manually switch to JDK 8:" + echo " sudo update-alternatives --config java" + echo " sudo update-alternatives --config javac" + echo " Then configure environment for JDK 8" + fi + else + echo " ✗ JDK 8 directory not found at expected location: $jdk8_path" + echo " Please manually locate and configure JDK 8" + fi + else + # For yum/dnf systems + echo " Please manually switch to JDK 8:" + echo " sudo alternatives --config java" + echo " sudo alternatives --config javac" + echo "" + echo " After switching, you can configure the environment." + + # Still create tron_java_env.sh for manual use + if configure_java_environment "8" "Linux" "$ARCH"; then + echo "Environment configuration completed for JDK 8." + else + echo "tron_java_env.sh has been created for manual use." + echo "After switching to JDK 8, run: source ./tron_java_env.sh" + fi + fi + else + echo "Error: Unable to install any JDK on $PKG_MANAGER" + exit 1 + fi + fi + + elif [[ "$ARCH" == "aarch64" ]] || [[ "$ARCH" == "arm64" ]]; then + echo ">>> Architecture is arm64/aarch64. Checking for JDK 17..." + set +e # Temporarily disable exit on error + check_java_version + local java_status=$? + set -e # Re-enable exit on error + + if [[ $java_status -eq 1 ]]; then + echo ">>> JDK 17 is already installed. Skipping installation." + else + if [[ $java_status -eq 0 ]]; then + echo ">>> Installing JDK 17 alongside existing JDK 8..." + elif [[ $java_status -eq 2 ]]; then + echo ">>> Installing JDK 17 alongside existing Java installation..." + else + echo ">>> Installing JDK 17..." + fi + if [[ "$PKG_MANAGER" == "apt-get" ]]; then + if install_first_available "17" openjdk-17-jdk; then + install_result=0 + else + install_result=$? + fi + else + if install_first_available "17" java-17-amazon-corretto-devel java-17-openjdk-devel; then + install_result=0 + else + install_result=$? + fi + fi + + if [[ $install_result -eq 0 ]]; then + # Use unified Java environment configuration + if configure_java_environment "17" "Linux" "$ARCH"; then + echo "Environment has been updated! Java 17 is now configured." + else + echo "Java 17 installed but environment not configured. You may need to set JAVA_HOME manually." + fi + elif [[ $install_result -eq 2 ]]; then + # JDK 17 package is installed but default version is different + # Need to switch to JDK 17 using update-alternatives + local actual_version=$(java -version 2>&1 | head -n 1 | cut -d'"' -f2) + echo ">>> JDK 17 package is installed, but system default is: $actual_version" + echo ">>> Switching system default to JDK 17 using update-alternatives..." + + # Try to switch to JDK 17 + if [[ "$PKG_MANAGER" == "apt-get" ]]; then + local jdk17_path="" + if [[ "$ARCH" == "aarch64" ]] || [[ "$ARCH" == "arm64" ]]; then + jdk17_path="/usr/lib/jvm/java-17-openjdk-arm64" + else + jdk17_path="/usr/lib/jvm/java-17-openjdk-amd64" + fi + + if [[ -d "$jdk17_path" ]]; then + echo " Found JDK 17 at: $jdk17_path" + # Set JDK 17 as default using update-alternatives + sudo update-alternatives --set java "$jdk17_path/bin/java" 2>/dev/null || \ + echo " Note: Could not auto-switch. Please run: sudo update-alternatives --config java" + + sudo update-alternatives --set javac "$jdk17_path/bin/javac" 2>/dev/null || \ + echo " Note: Could not auto-switch javac. Please run: sudo update-alternatives --config javac" + + # Verify the switch + local new_version=$(java -version 2>&1 | head -n 1 | cut -d'"' -f2) + if [[ "$new_version" =~ ^17\. ]]; then + echo " ✓ Successfully switched to JDK 17: $new_version" + # Now configure environment for JDK 17 + if configure_java_environment "17" "Linux" "$ARCH"; then + echo "Environment has been updated! Java 17 is now configured." + else + echo "Java 17 is active but environment not configured. You may need to set JAVA_HOME manually." + fi + else + echo " ✗ Auto-switch failed. Current version: $new_version" + echo " Please manually switch to JDK 17:" + echo " sudo update-alternatives --config java" + echo " sudo update-alternatives --config javac" + echo " Then configure environment for JDK 17" + fi + else + echo " ✗ JDK 17 directory not found at expected location: $jdk17_path" + echo " Please manually locate and configure JDK 17" + fi + else + # For yum/dnf systems + echo " Please manually switch to JDK 17:" + echo " sudo alternatives --config java" + echo " sudo alternatives --config javac" + echo "" + echo " After switching, you can configure the environment." + + # Still create tron_java_env.sh for manual use + if configure_java_environment "17" "Linux" "$ARCH"; then + echo "Environment configuration completed for JDK 17." + else + echo "tron_java_env.sh has been created for manual use." + echo "After switching to JDK 17, run: source ./tron_java_env.sh" + fi + fi + else + echo "Error: Unable to install any JDK on $PKG_MANAGER" + exit 1 + fi + fi + + else + echo "Error: Unsupported architecture for Linux script: $ARCH" + exit 1 + fi +} + +if [[ "$OS" == "Darwin" ]]; then + install_macos +elif [[ "$OS" == "Linux" ]]; then + install_linux +else + echo "Error: Unsupported Operating System: $OS" + exit 1 +fi + +echo "----------------------------------------" +echo "Installation completed successfully!" +echo "" +echo ">>> Verification Commands:" +echo " git --version" +echo " java -version" +echo "" + +# Verify that Java is actually working +echo ">>> Verifying Java installation..." +if command -v java &> /dev/null; then + echo " Java command found: $(which java)" + if java -version &> /dev/null; then + echo " Java version: $(java -version 2>&1 | head -n 1)" + else + echo " ✗ Java command exists but cannot run properly" + echo " Please run: source ./tron_java_env.sh" + fi +else + echo " ✗ Java command not found in PATH" + echo " Please run: source ./tron_java_env.sh" + echo " If that doesn't work, check the Java environment configuration above." +fi + +echo "" +echo ">>> Troubleshooting:" +echo " - If 'java -version' shows incorrect version, run: source ./tron_java_env.sh" +echo " - For permanent configuration, follow the instructions shown above." +echo "" +echo "" \ No newline at end of file diff --git a/jitpack.yml b/jitpack.yml index f951e400136..4065d43092c 100644 --- a/jitpack.yml +++ b/jitpack.yml @@ -1,2 +1,2 @@ install: - - ./gradlew clean -xtest -xlint -xcheck -PbinaryRelease=false install \ No newline at end of file + - ./gradlew clean -xtest -xlint -xcheck -PbinaryRelease=false publishToMavenLocal diff --git a/platform/build.gradle b/platform/build.gradle new file mode 100644 index 00000000000..a94aad3cf17 --- /dev/null +++ b/platform/build.gradle @@ -0,0 +1,17 @@ +description = "platform – a distributed consensus arithmetic for blockchain." + +sourceSets { + main { + java.srcDirs = rootProject.archInfo.sourceSets.main.java.srcDirs + } + test { + java.srcDirs = rootProject.archInfo.sourceSets.test.java.srcDirs + } +} + +dependencies { + api group: 'org.fusesource.leveldbjni', name: 'leveldbjni-all', version: '1.8' + api group: 'org.rocksdb', name: 'rocksdbjni', version: "${rootProject.archInfo.requires.RocksdbVersion}" + api group: 'commons-io', name: 'commons-io', version: '2.18.0' + api 'io.github.tronprotocol:zksnark-java-sdk:1.0.0' exclude(group: 'commons-io', module: 'commons-io') +} diff --git a/platform/src/main/java/arm/org/tron/common/math/MathWrapper.java b/platform/src/main/java/arm/org/tron/common/math/MathWrapper.java new file mode 100644 index 00000000000..12395dffcea --- /dev/null +++ b/platform/src/main/java/arm/org/tron/common/math/MathWrapper.java @@ -0,0 +1,254 @@ +package org.tron.common.math; + +import java.util.Collections; +import java.util.HashMap; +import java.util.Map; +import java.util.Objects; + +/** + * This class is deprecated and should not be used in new code, + * for cross-platform consistency, please use {@link StrictMathWrapper} instead, + * especially for floating-point calculations. + */ +@Deprecated +public class MathWrapper { + + private static final Map powData = Collections.synchronizedMap(new HashMap<>()); + private static final String EXPONENT = "3f40624dd2f1a9fc"; // 1/2000 = 0.0005 + + public static double pow(double a, double b) { + double strictResult = StrictMath.pow(a, b); + return powData.getOrDefault(new PowData(a, b), strictResult); + } + + /** + * This static block is used to initialize the data map. + */ + static { + // init main-net pow data start + addPowData("3ff0192278704be3", EXPONENT, "3ff000033518c576"); // 4137160(block) + addPowData("3ff000002fc6a33f", EXPONENT, "3ff0000000061d86"); // 4065476 + addPowData("3ff00314b1e73ecf", EXPONENT, "3ff0000064ea3ef8"); // 4071538 + addPowData("3ff0068cd52978ae", EXPONENT, "3ff00000d676966c"); // 4109544 + addPowData("3ff0032fda05447d", EXPONENT, "3ff0000068636fe0"); // 4123826 + addPowData("3ff00051c09cc796", EXPONENT, "3ff000000a76c20e"); // 4166806 + addPowData("3ff00bef8115b65d", EXPONENT, "3ff0000186893de0"); // 4225778 + addPowData("3ff009b0b2616930", EXPONENT, "3ff000013d27849e"); // 4251796 + addPowData("3ff00364ba163146", EXPONENT, "3ff000006f26a9dc"); // 4257157 + addPowData("3ff019be4095d6ae", EXPONENT, "3ff0000348e9f02a"); // 4260583 + addPowData("3ff0123e52985644", EXPONENT, "3ff0000254797fd0"); // 4367125 + addPowData("3ff0126d052860e2", EXPONENT, "3ff000025a6cde26"); // 4402197 + addPowData("3ff0001632cccf1b", EXPONENT, "3ff0000002d76406"); // 4405788 + addPowData("3ff0000965922b01", EXPONENT, "3ff000000133e966"); // 4490332 + addPowData("3ff00005c7692d61", EXPONENT, "3ff0000000bd5d34"); // 4499056 + addPowData("3ff015cba20ec276", EXPONENT, "3ff00002c84cef0e"); // 4518035 + addPowData("3ff00002f453d343", EXPONENT, "3ff000000060cf4e"); // 4533215 + addPowData("3ff006ea73f88946", EXPONENT, "3ff00000e26d4ea2"); // 4647814 + addPowData("3ff00a3632db72be", EXPONENT, "3ff000014e3382a6"); // 4766695 + addPowData("3ff000c0e8df0274", EXPONENT, "3ff0000018b0aeb2"); // 4771494 + addPowData("3ff00015c8f06afe", EXPONENT, "3ff0000002c9d73e"); // 4793587 + addPowData("3ff00068def18101", EXPONENT, "3ff000000d6c3cac"); // 4801947 + addPowData("3ff01349f3ac164b", EXPONENT, "3ff000027693328a"); // 4916843 + addPowData("3ff00e86a7859088", EXPONENT, "3ff00001db256a52"); // 4924111 + addPowData("3ff00000c2a51ab7", EXPONENT, "3ff000000018ea20"); // 5098864 + addPowData("3ff020fb74e9f170", EXPONENT, "3ff00004346fbfa2"); // 5133963 + addPowData("3ff00001ce277ce7", EXPONENT, "3ff00000003b27dc"); // 5139389 + addPowData("3ff005468a327822", EXPONENT, "3ff00000acc20750"); // 5151258 + addPowData("3ff00006666f30ff", EXPONENT, "3ff0000000d1b80e"); // 5185021 + addPowData("3ff000045a0b2035", EXPONENT, "3ff00000008e98e6"); // 5295829 + addPowData("3ff00e00380e10d7", EXPONENT, "3ff00001c9ff83c8"); // 5380897 + addPowData("3ff00c15de2b0d5e", EXPONENT, "3ff000018b6eaab6"); // 5400886 + addPowData("3ff00042afe6956a", EXPONENT, "3ff0000008892244"); // 5864127 + addPowData("3ff0005b7357c2d4", EXPONENT, "3ff000000bb48572"); // 6167339 + addPowData("3ff00033d5ab51c8", EXPONENT, "3ff0000006a279c8"); // 6240974 + addPowData("3ff0000046d74585", EXPONENT, "3ff0000000091150"); // 6279093 + addPowData("3ff0010403f34767", EXPONENT, "3ff0000021472146"); // 6428736 + addPowData("3ff00496fe59bc98", EXPONENT, "3ff000009650a4ca"); // 6432355,6493373 + addPowData("3ff0012e43815868", EXPONENT, "3ff0000026af266e"); // 6555029 + addPowData("3ff00021f6080e3c", EXPONENT, "3ff000000458d16a"); // 7092933 + addPowData("3ff000489c0f28bd", EXPONENT, "3ff00000094b3072"); // 7112412 + addPowData("3ff00009d3df2e9c", EXPONENT, "3ff00000014207b4"); // 7675535 + addPowData("3ff000def05fa9c8", EXPONENT, "3ff000001c887cdc"); // 7860324 + addPowData("3ff0013bca543227", EXPONENT, "3ff00000286a42d2"); // 8292427 + addPowData("3ff0021a2f14a0ee", EXPONENT, "3ff0000044deb040"); // 8517311 + addPowData("3ff0002cc166be3c", EXPONENT, "3ff0000005ba841e"); // 8763101 + addPowData("3ff0000cc84e613f", EXPONENT, "3ff0000001a2da46"); // 9269124 + addPowData("3ff000057b83c83f", EXPONENT, "3ff0000000b3a640"); // 9631452 + // init main-net pow data end + // add pow data + } + + private static void addPowData(String a, String b, String ret) { + powData.put(new PowData(hexToDouble(a), hexToDouble(b)), hexToDouble(ret)); + } + + private static double hexToDouble(String input) { + // Convert the hex string to a long + long hexAsLong = Long.parseLong(input, 16); + // and then convert the long to a double + return Double.longBitsToDouble(hexAsLong); + } + + private static class PowData { + final double a; + final double b; + + public PowData(double a, double b) { + this.a = a; + this.b = b; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + PowData powData = (PowData) o; + return Double.compare(powData.a, a) == 0 && Double.compare(powData.b, b) == 0; + } + + @Override + public int hashCode() { + return Objects.hash(a, b); + } + } + + /** + * *** methods are same as {@link java.lang.Math} methods, guaranteed by the call start *** + */ + + /** + * finally calls {@link java.lang.Math#addExact(long, long)} + */ + + public static long addExact(long x, long y) { + return StrictMath.addExact(x, y); + } + + /** + * finally calls {@link java.lang.Math#addExact(int, int)} + */ + + public static int addExact(int x, int y) { + return StrictMath.addExact(x, y); + } + + /** + * finally calls {@link java.lang.Math#subtractExact(long, long)} + */ + + public static long subtractExact(long x, long y) { + return StrictMath.subtractExact(x, y); + } + + /** + * finally calls {@link java.lang.Math#floorMod(long, long)} + */ + public static long multiplyExact(long x, long y) { + return StrictMath.multiplyExact(x, y); + } + + public static long multiplyExact(long x, int y) { + return multiplyExact(x, (long) y); + } + + public static int multiplyExact(int x, int y) { + return StrictMath.multiplyExact(x, y); + } + + /** + * finally calls {@link java.lang.Math#floorDiv(long, long)} + */ + public static long floorDiv(long x, long y) { + return StrictMath.floorDiv(x, y); + } + + public static long floorDiv(long x, int y) { + return floorDiv(x, (long) y); + } + + /** + * finally calls {@link java.lang.Math#min(int, int)} + */ + public static int min(int a, int b) { + return StrictMath.min(a, b); + } + + /** + * finally calls {@link java.lang.Math#min(long, long)} + */ + public static long min(long a, long b) { + return StrictMath.min(a, b); + } + + /** + * finally calls {@link java.lang.Math#max(int, int)} + */ + public static int max(int a, int b) { + return StrictMath.max(a, b); + } + + /** + * finally calls {@link java.lang.Math#max(long, long)} + */ + public static long max(long a, long b) { + return StrictMath.max(a, b); + } + + /** + * finally calls {@link java.lang.Math#round(float)} + */ + public static int round(float a) { + return StrictMath.round(a); + } + + /** + * finally calls {@link java.lang.Math#round(double)} + */ + public static long round(double a) { + return StrictMath.round(a); + } + + /** + * finally calls {@link java.lang.Math#signum(double)} + */ + public static double signum(double d) { + return StrictMath.signum(d); + } + + /** + * finally calls {@link java.lang.Math#signum(float)} + */ + public static long abs(long a) { + return StrictMath.abs(a); + } + + /** + * *** methods are same as {@link java.lang.Math} methods, guaranteed by the call end *** + */ + + /** + * *** methods are same as {@link java.lang.Math} methods by mathematical algorithms*** + * / + + + /** + * mathematical integer: ceil(i) = floor(i) = i + * @return the smallest (closest to negative infinity) double value that is greater + * than or equal to the argument and is equal to a mathematical integer. + */ + public static double ceil(double a) { + return StrictMath.ceil(a); + } + + /** + * *** methods are no matters *** + */ + public static double random() { + return StrictMath.random(); + } + +} diff --git a/platform/src/main/java/arm/org/tron/common/utils/MarketOrderPriceComparatorForRocksDB.java b/platform/src/main/java/arm/org/tron/common/utils/MarketOrderPriceComparatorForRocksDB.java new file mode 100644 index 00000000000..26c246faf0e --- /dev/null +++ b/platform/src/main/java/arm/org/tron/common/utils/MarketOrderPriceComparatorForRocksDB.java @@ -0,0 +1,32 @@ +package org.tron.common.utils; + +import java.nio.ByteBuffer; +import org.rocksdb.AbstractComparator; +import org.rocksdb.ComparatorOptions; + +public class MarketOrderPriceComparatorForRocksDB extends AbstractComparator { + + public MarketOrderPriceComparatorForRocksDB(final ComparatorOptions copt) { + super(copt); + } + + @Override + public String name() { + return "MarketOrderPriceComparator"; + } + + @Override + public int compare(final ByteBuffer a, final ByteBuffer b) { + return MarketComparator.comparePriceKey(convertDataToBytes(a), convertDataToBytes(b)); + } + + /** + * DirectSlice.data().array will throw UnsupportedOperationException. + * */ + public byte[] convertDataToBytes(ByteBuffer buf) { + byte[] bytes = new byte[buf.remaining()]; + buf.get(bytes); + return bytes; + } + +} diff --git a/platform/src/main/java/common/org/tron/common/arch/Arch.java b/platform/src/main/java/common/org/tron/common/arch/Arch.java new file mode 100644 index 00000000000..f115d1f07c2 --- /dev/null +++ b/platform/src/main/java/common/org/tron/common/arch/Arch.java @@ -0,0 +1,91 @@ +package org.tron.common.arch; + +import lombok.extern.slf4j.Slf4j; + +@Slf4j(topic = "arch") +public final class Arch { + + private Arch() { + } + + public static String withAll() { + final StringBuilder info = new StringBuilder(); + info.append("os.name").append(": ").append(getOsName()).append("\n"); + info.append("os.arch").append(": ").append(getOsArch()).append("\n"); + info.append("bit.model").append(": ").append(getBitModel()).append("\n"); + info.append("java.version").append(": ").append(javaVersion()).append("\n"); + info.append("java.specification.version").append(": ").append(javaSpecificationVersion()) + .append("\n"); + info.append("java.vendor").append(": ").append(javaVendor()).append("\n"); + return info.toString(); + } + + public static String getOsName() { + return System.getProperty("os.name").toLowerCase().trim(); + + } + public static String getOsArch() { + return System.getProperty("os.arch").toLowerCase().trim(); + } + + public static int getBitModel() { + String prop = System.getProperty("sun.arch.data.model"); + if (prop == null) { + prop = System.getProperty("com.ibm.vm.bitmode"); + } + if (prop != null) { + return Integer.parseInt(prop); + } + // GraalVM support, see https://github.com/fusesource/jansi/issues/162 + String arch = System.getProperty("os.arch"); + if (arch.endsWith("64") && "Substrate VM".equals(System.getProperty("java.vm.name"))) { + return 64; + } + return -1; // we don't know... + } + + public static String javaVersion() { + return System.getProperty("java.version").toLowerCase().trim(); + } + + public static String javaSpecificationVersion() { + return System.getProperty("java.specification.version").toLowerCase().trim(); + } + + public static String javaVendor() { + return System.getProperty("java.vendor").toLowerCase().trim(); + } + + public static boolean isArm64() { + String osArch = getOsArch(); + return osArch.contains("arm64") || osArch.contains("aarch64"); + } + + public static boolean isX86() { + return !isArm64(); + } + + public static boolean isJava8() { + return javaSpecificationVersion().equals("1.8"); + } + + public static boolean isJava17() { + return javaSpecificationVersion().equals("17"); + } + + public static void throwIfUnsupportedJavaVersion() { + if ((isX86() && !isJava8()) || (isArm64() && !isJava17())) { + logger.info(withAll()); + throw new UnsupportedOperationException(String.format( + "Java %s is required for %s architecture. Detected version %s", isX86() ? "1.8" : "17", + getOsArch(), javaSpecificationVersion())); + } + } + + public static void throwIfUnsupportedArm64Exception(String message) { + if (isArm64()) { + throw new UnsupportedOperationException( + message + ": unsupported on " + getOsArch() + " architecture"); + } + } +} diff --git a/plugins/src/main/java/org/tron/plugins/utils/MarketUtils.java b/platform/src/main/java/common/org/tron/common/utils/MarketComparator.java similarity index 50% rename from plugins/src/main/java/org/tron/plugins/utils/MarketUtils.java rename to platform/src/main/java/common/org/tron/common/utils/MarketComparator.java index dbd578a59a3..c2742d4d10b 100644 --- a/plugins/src/main/java/org/tron/plugins/utils/MarketUtils.java +++ b/platform/src/main/java/common/org/tron/common/utils/MarketComparator.java @@ -1,71 +1,10 @@ -package org.tron.plugins.utils; +package org.tron.common.utils; import java.math.BigInteger; -import org.tron.plugins.utils.ByteArray; -public class MarketUtils { +public class MarketComparator { - public static final int TOKEN_ID_LENGTH = ByteArray - .fromString(Long.toString(Long.MAX_VALUE)).length; // 19 - - - - /** - * In order to avoid the difference between the data of same key stored and fetched by hashMap and - * levelDB, when creating the price key, we will find the GCD (Greatest Common Divisor) of - * sellTokenQuantity and buyTokenQuantity. - */ - public static byte[] createPairPriceKey(byte[] sellTokenId, byte[] buyTokenId, - long sellTokenQuantity, long buyTokenQuantity) { - - byte[] sellTokenQuantityBytes; - byte[] buyTokenQuantityBytes; - - // cal the GCD - long gcd = findGCD(sellTokenQuantity, buyTokenQuantity); - if (gcd == 0) { - sellTokenQuantityBytes = ByteArray.fromLong(sellTokenQuantity); - buyTokenQuantityBytes = ByteArray.fromLong(buyTokenQuantity); - } else { - sellTokenQuantityBytes = ByteArray.fromLong(sellTokenQuantity / gcd); - buyTokenQuantityBytes = ByteArray.fromLong(buyTokenQuantity / gcd); - } - - return doCreatePairPriceKey(sellTokenId, buyTokenId, - sellTokenQuantityBytes, buyTokenQuantityBytes); - } - - public static long findGCD(long number1, long number2) { - if (number1 == 0 || number2 == 0) { - return 0; - } - return calGCD(number1, number2); - } - - private static long calGCD(long number1, long number2) { - if (number2 == 0) { - return number1; - } - return calGCD(number2, number1 % number2); - } - - - private static byte[] doCreatePairPriceKey(byte[] sellTokenId, byte[] buyTokenId, - byte[] sellTokenQuantity, byte[] buyTokenQuantity) { - byte[] result = new byte[TOKEN_ID_LENGTH + TOKEN_ID_LENGTH - + sellTokenQuantity.length + buyTokenQuantity.length]; - - System.arraycopy(sellTokenId, 0, result, 0, sellTokenId.length); - System.arraycopy(buyTokenId, 0, result, TOKEN_ID_LENGTH, buyTokenId.length); - System.arraycopy(sellTokenQuantity, 0, result, - TOKEN_ID_LENGTH + TOKEN_ID_LENGTH, - sellTokenQuantity.length); - System.arraycopy(buyTokenQuantity, 0, result, - TOKEN_ID_LENGTH + TOKEN_ID_LENGTH + buyTokenQuantity.length, - buyTokenQuantity.length); - - return result; - } + public static final int TOKEN_ID_LENGTH = Long.toString(Long.MAX_VALUE).getBytes().length; // 19 public static int comparePriceKey(byte[] o1, byte[] o2) { @@ -76,7 +15,7 @@ public static int comparePriceKey(byte[] o1, byte[] o2) { System.arraycopy(o1, 0, pair1, 0, TOKEN_ID_LENGTH * 2); System.arraycopy(o2, 0, pair2, 0, TOKEN_ID_LENGTH * 2); - int pairResult = ByteArray.compareUnsigned(pair1, pair2); + int pairResult = compareUnsigned(pair1, pair2); if (pairResult != 0) { return pairResult; } @@ -100,10 +39,10 @@ public static int comparePriceKey(byte[] o1, byte[] o2) { System.arraycopy(o2, TOKEN_ID_LENGTH + TOKEN_ID_LENGTH + longByteNum, getBuyTokenQuantity2, 0, longByteNum); - long sellTokenQuantity1 = ByteArray.toLong(getSellTokenQuantity1); - long buyTokenQuantity1 = ByteArray.toLong(getBuyTokenQuantity1); - long sellTokenQuantity2 = ByteArray.toLong(getSellTokenQuantity2); - long buyTokenQuantity2 = ByteArray.toLong(getBuyTokenQuantity2); + long sellTokenQuantity1 = toLong(getSellTokenQuantity1); + long buyTokenQuantity1 = toLong(getBuyTokenQuantity1); + long sellTokenQuantity2 = toLong(getSellTokenQuantity2); + long buyTokenQuantity2 = toLong(getBuyTokenQuantity2); if ((sellTokenQuantity1 == 0 || buyTokenQuantity1 == 0) && (sellTokenQuantity2 == 0 || buyTokenQuantity2 == 0)) { @@ -145,4 +84,41 @@ public static int comparePrice(long price1SellQuantity, long price1BuyQuantity, return price1BuyQuantityBI.multiply(price2SellQuantityBI) .compareTo(price2BuyQuantityBI.multiply(price1SellQuantityBI)); } + + /** + * copy from org.bouncycastle.util.Arrays.compareUnsigned + */ + private static int compareUnsigned(byte[] a, byte[] b) { + if (a == b) { + return 0; + } + if (a == null) { + return -1; + } + if (b == null) { + return 1; + } + int minLen = StrictMath.min(a.length, b.length); + for (int i = 0; i < minLen; ++i) { + int aVal = a[i] & 0xFF; + int bVal = b[i] & 0xFF; + if (aVal < bVal) { + return -1; + } + if (aVal > bVal) { + return 1; + } + } + if (a.length < b.length) { + return -1; + } + if (a.length > b.length) { + return 1; + } + return 0; + } + + public static long toLong(byte[] b) { + return (b == null || b.length == 0) ? 0 : new BigInteger(1, b).longValue(); + } } diff --git a/chainbase/src/main/java/org/tron/common/utils/MarketOrderPriceComparatorForLevelDB.java b/platform/src/main/java/common/org/tron/common/utils/MarketOrderPriceComparatorForLevelDB.java similarity index 87% rename from chainbase/src/main/java/org/tron/common/utils/MarketOrderPriceComparatorForLevelDB.java rename to platform/src/main/java/common/org/tron/common/utils/MarketOrderPriceComparatorForLevelDB.java index c4c519f68f1..efb7219b211 100644 --- a/chainbase/src/main/java/org/tron/common/utils/MarketOrderPriceComparatorForLevelDB.java +++ b/platform/src/main/java/common/org/tron/common/utils/MarketOrderPriceComparatorForLevelDB.java @@ -1,7 +1,5 @@ package org.tron.common.utils; -import org.tron.core.capsule.utils.MarketUtils; - public class MarketOrderPriceComparatorForLevelDB implements org.iq80.leveldb.DBComparator { @Override @@ -26,7 +24,7 @@ public byte[] findShortSuccessor(byte[] key) { */ @Override public int compare(byte[] o1, byte[] o2) { - return MarketUtils.comparePriceKey(o1, o2); + return MarketComparator.comparePriceKey(o1, o2); } } diff --git a/common/src/main/java/org/tron/common/math/MathWrapper.java b/platform/src/main/java/x86/org/tron/common/math/MathWrapper.java similarity index 100% rename from common/src/main/java/org/tron/common/math/MathWrapper.java rename to platform/src/main/java/x86/org/tron/common/math/MathWrapper.java diff --git a/chainbase/src/main/java/org/tron/common/utils/MarketOrderPriceComparatorForRockDB.java b/platform/src/main/java/x86/org/tron/common/utils/MarketOrderPriceComparatorForRocksDB.java similarity index 70% rename from chainbase/src/main/java/org/tron/common/utils/MarketOrderPriceComparatorForRockDB.java rename to platform/src/main/java/x86/org/tron/common/utils/MarketOrderPriceComparatorForRocksDB.java index d3812f0f6bc..be406ff658d 100644 --- a/chainbase/src/main/java/org/tron/common/utils/MarketOrderPriceComparatorForRockDB.java +++ b/platform/src/main/java/x86/org/tron/common/utils/MarketOrderPriceComparatorForRocksDB.java @@ -3,11 +3,10 @@ import org.rocksdb.ComparatorOptions; import org.rocksdb.DirectSlice; import org.rocksdb.util.DirectBytewiseComparator; -import org.tron.core.capsule.utils.MarketUtils; -public class MarketOrderPriceComparatorForRockDB extends DirectBytewiseComparator { +public class MarketOrderPriceComparatorForRocksDB extends DirectBytewiseComparator { - public MarketOrderPriceComparatorForRockDB(final ComparatorOptions copt) { + public MarketOrderPriceComparatorForRocksDB(final ComparatorOptions copt) { super(copt); } @@ -18,7 +17,7 @@ public String name() { @Override public int compare(final DirectSlice a, final DirectSlice b) { - return MarketUtils.comparePriceKey(convertDataToBytes(a), convertDataToBytes(b)); + return MarketComparator.comparePriceKey(convertDataToBytes(a), convertDataToBytes(b)); } /** diff --git a/plugins/README.md b/plugins/README.md index 0db6f2e6143..db25811882f 100644 --- a/plugins/README.md +++ b/plugins/README.md @@ -2,7 +2,7 @@ This package contains a set of tools for TRON, the followings are the documentation for each tool. -## DB Archive +## DB Archive(Requires x86 + LevelDB) DB archive provides the ability to reformat the manifest according to the current `database`, parameters are compatible with the previous `ArchiveManifest`. @@ -26,7 +26,7 @@ DB archive provides the ability to reformat the manifest according to the curren ``` -## DB Convert +## DB Convert(Requires x86 + LevelDB) DB convert provides a helper which can convert LevelDB data to RocksDB data, parameters are compatible with previous `DBConvert`. @@ -34,15 +34,13 @@ DB convert provides a helper which can convert LevelDB data to RocksDB data, par - ``: Input path for leveldb, default: output-directory/database. - ``: Output path for rocksdb, default: output-directory-dst/database. -- `--safe`: In safe mode, read data from leveldb then put into rocksdb, it's a very time-consuming procedure. If not, just change engine.properties from leveldb to rocksdb, rocksdb - is compatible with leveldb for the current version. This may not be the case in the future, default: false. - `-h | --help`: Provide the help info. ### Examples: ```shell script # full command - java -jar Toolkit.jar db convert [-h] [--safe] + java -jar Toolkit.jar db convert [-h] # examples java -jar Toolkit.jar db convert output-directory/database /tmp/database ``` @@ -66,7 +64,7 @@ DB copy provides a helper which can copy LevelDB or RocksDB data quickly on the java -jar Toolkit.jar db cp output-directory/database /tmp/databse ``` -## DB Lite +## DB Lite(LevelDB unavailable on ARM) DB lite provides lite database, parameters are compatible with previous `LiteFullNodeTool`. @@ -134,7 +132,7 @@ Execute move command. java -jar Toolkit.jar db mv -c main_net_config.conf -d /data/tron/output-directory ``` -## DB Root +## DB Root(LevelDB unavailable on ARM) DB root provides a helper which can compute merkle root for tiny db. diff --git a/plugins/build.gradle b/plugins/build.gradle index 01afaa01708..e03e9a7c49a 100644 --- a/plugins/build.gradle +++ b/plugins/build.gradle @@ -20,6 +20,15 @@ configurations.getByName('checkstyleConfig') { transitive = false } +sourceSets { + main { + java.srcDirs = rootProject.archInfo.sourceSets.main.java.srcDirs + } + test { + java.srcDirs = rootProject.archInfo.sourceSets.test.java.srcDirs + } +} + dependencies { //local libraries implementation fileTree(dir: 'libs', include: '*.jar') @@ -28,10 +37,19 @@ dependencies { implementation group: 'info.picocli', name: 'picocli', version: '4.6.3' implementation group: 'com.typesafe', name: 'config', version: '1.3.2' implementation group: 'me.tongfei', name: 'progressbar', version: '0.9.3' - implementation group: 'org.bouncycastle', name: 'bcprov-jdk15on', version: '1.69' - implementation group: 'org.rocksdb', name: 'rocksdbjni', version: '5.15.10' - implementation 'io.github.tronprotocol:leveldbjni-all:1.18.2' - implementation 'io.github.tronprotocol:leveldb:1.18.2' + implementation group: 'org.bouncycastle', name: 'bcprov-jdk18on', version: '1.79' + if (rootProject.archInfo.isArm64) { + testRuntimeOnly group: 'org.fusesource.hawtjni', name: 'hawtjni-runtime', version: '1.18' // for test + implementation project(":platform") + } else { + implementation project(":platform"), { + exclude(group: 'org.fusesource.leveldbjni', module: 'leveldbjni-all') + exclude(group: 'io.github.tronprotocol', module: 'zksnark-java-sdk') + exclude(group: 'commons-io', module: 'commons-io') + } + implementation 'io.github.tronprotocol:leveldbjni-all:1.18.2' + implementation 'io.github.tronprotocol:leveldb:1.18.2' + } implementation project(":protocol") } @@ -76,6 +94,17 @@ test { destinationFile = file("../framework/build/jacoco/jacocoTest1.exec") classDumpDir = file("$buildDir/jacoco/classpathdumps") } + + if (rootProject.archInfo.isArm64) { + exclude 'org/tron/plugins/leveldb/**' + filter { + excludeTestsMatching '*.*leveldb*' + excludeTestsMatching '*.*Leveldb*' + excludeTestsMatching '*.*LevelDB*' + excludeTestsMatching '*.*LevelDb*' + excludeTestsMatching '*.*Archive*' + } + } } jacocoTestReport { @@ -94,7 +123,7 @@ def binaryRelease(taskName, jarName, mainClass) { from(sourceSets.main.output) { include "/**" } - dependsOn project(':protocol').jar // explicit_dependency + dependsOn (project(':protocol').jar, project(':platform').jar) // explicit_dependency from { configurations.runtimeClasspath.collect { // https://docs.gradle.org/current/userguide/upgrading_version_6.html#changes_6.3 it.isDirectory() ? it : zipTree(it) @@ -127,7 +156,7 @@ def createScript(project, mainClass, name) { } } } -applicationDistribution.from("../gradle/java-tron.vmoptions") { +applicationDistribution.from(rootProject.archInfo.VMOptions) { into "bin" } createScript(project, 'org.tron.plugins.ArchiveManifest', 'ArchiveManifest') @@ -157,4 +186,4 @@ task copyToParent(type: Copy) { -build.finalizedBy(copyToParent) \ No newline at end of file +build.finalizedBy(copyToParent) diff --git a/plugins/src/main/java/arm/org/tron/plugins/ArchiveManifest.java b/plugins/src/main/java/arm/org/tron/plugins/ArchiveManifest.java new file mode 100644 index 00000000000..b7848cf4c6f --- /dev/null +++ b/plugins/src/main/java/arm/org/tron/plugins/ArchiveManifest.java @@ -0,0 +1,29 @@ +package org.tron.plugins; + +import lombok.extern.slf4j.Slf4j; +import org.tron.common.arch.Arch; + +/** + * ARM architecture only supports RocksDB, + * which does not require manifest rebuilding (manifest rebuilding is a LevelDB-only feature). + * This command is not supported but retained for compatibility. + **/ +@Slf4j(topic = "archive") +public class ArchiveManifest { + + public static void main(String[] args) { + int exitCode = run(args); + System.exit(exitCode); + } + + public static int run(String[] args) { + String tips = String.format( + "%s architecture only supports RocksDB, which does not require manifest rebuilding " + + "(manifest rebuilding is a LevelDB-only feature).", + Arch.getOsArch()); + System.out.println(tips); + logger.warn(tips); + return 0; + } + +} diff --git a/plugins/src/main/java/arm/org/tron/plugins/DbArchive.java b/plugins/src/main/java/arm/org/tron/plugins/DbArchive.java new file mode 100644 index 00000000000..03c52f33f02 --- /dev/null +++ b/plugins/src/main/java/arm/org/tron/plugins/DbArchive.java @@ -0,0 +1,50 @@ +package org.tron.plugins; + +import java.util.concurrent.Callable; +import lombok.extern.slf4j.Slf4j; +import org.tron.common.arch.Arch; +import picocli.CommandLine; +import picocli.CommandLine.Option; + +/** + * ARM architecture only supports RocksDB, + * which does not require manifest rebuilding (manifest rebuilding is a LevelDB-only feature). + * This command is not supported but retained for compatibility. + **/ +@Slf4j(topic = "archive") +@CommandLine.Command(name = "archive", description = "A helper to rewrite leveldb manifest.") +public class DbArchive implements Callable { + + @CommandLine.Spec + CommandLine.Model.CommandSpec spec; + @Option(names = {"-d", "--database-directory"}, + defaultValue = "output-directory/database", + description = "java-tron database directory. Default: ${DEFAULT-VALUE}") + private String databaseDirectory; + + @Option(names = {"-b", "--batch-size"}, + defaultValue = "80000", + description = "deal manifest batch size. Default: ${DEFAULT-VALUE}") + private int maxBatchSize; + + @Option(names = {"-m", "--manifest-size"}, + defaultValue = "0", + description = "manifest min size(M) to archive. Default: ${DEFAULT-VALUE}") + private int maxManifestSize; + + @Option(names = {"-h", "--help"}) + private boolean help; + + + @Override + public Integer call() throws Exception { + String tips = String.format( + "%s architecture only supports RocksDB, which does not require manifest rebuilding " + + "(manifest rebuilding is a LevelDB-only feature).", + Arch.getOsArch()); + spec.commandLine().getErr().println(spec.commandLine().getColorScheme().errorText(tips)); + logger.warn(tips); + return 0; + } + +} diff --git a/plugins/src/main/java/org/tron/plugins/Db.java b/plugins/src/main/java/common/org/tron/plugins/Db.java similarity index 100% rename from plugins/src/main/java/org/tron/plugins/Db.java rename to plugins/src/main/java/common/org/tron/plugins/Db.java diff --git a/plugins/src/main/java/org/tron/plugins/DbConvert.java b/plugins/src/main/java/common/org/tron/plugins/DbConvert.java similarity index 89% rename from plugins/src/main/java/org/tron/plugins/DbConvert.java rename to plugins/src/main/java/common/org/tron/plugins/DbConvert.java index a75b235bbcf..bcf6e1e7afc 100644 --- a/plugins/src/main/java/org/tron/plugins/DbConvert.java +++ b/plugins/src/main/java/common/org/tron/plugins/DbConvert.java @@ -20,6 +20,7 @@ import org.rocksdb.RocksDBException; import org.rocksdb.RocksIterator; import org.rocksdb.Status; +import org.tron.common.arch.Arch; import org.tron.plugins.utils.DBUtils; import org.tron.plugins.utils.FileUtils; import picocli.CommandLine; @@ -49,20 +50,19 @@ public class DbConvert implements Callable { description = "Output path for rocksdb. Default: ${DEFAULT-VALUE}") private File dest; - @CommandLine.Option(names = {"--safe"}, - description = "In safe mode, read data from leveldb then put rocksdb." - + "If not, just change engine.properties from leveldb to rocksdb," - + "rocksdb is compatible with leveldb for current version." - + "This may not be the case in the future." - + "Default: ${DEFAULT-VALUE}") - private boolean safe; - @CommandLine.Option(names = {"-h", "--help"}) private boolean help; @Override public Integer call() throws Exception { + if (Arch.isArm64()) { + String tips = String.format("This command is not supported on %s architecture.", + Arch.getOsArch()); + spec.commandLine().getErr().println(spec.commandLine().getColorScheme().errorText(tips)); + logger.error(tips); + return 0; + } if (help) { spec.commandLine().usage(System.out); return 0; @@ -95,12 +95,12 @@ public Integer call() throws Exception { final long time = System.currentTimeMillis(); List services = new ArrayList<>(); files.forEach(f -> services.add( - new DbConverter(src.getPath(), dest.getPath(), f.getName(), safe))); + new DbConverter(src.getPath(), dest.getPath(), f.getName()))); cpList.forEach(f -> services.add( new DbConverter( Paths.get(src.getPath(), DBUtils.CHECKPOINT_DB_V2).toString(), Paths.get(dest.getPath(), DBUtils.CHECKPOINT_DB_V2).toString(), - f.getName(), safe))); + f.getName()))); List fails = ProgressBar.wrap(services.stream(), "convert task").parallel().map( dbConverter -> { try { @@ -140,15 +140,12 @@ static class DbConverter implements Converter { private long srcDbValueSum = 0L; private long dstDbValueSum = 0L; - private boolean safe; - - public DbConverter(String srcDir, String dstDir, String name, boolean safe) { + public DbConverter(String srcDir, String dstDir, String name) { this.srcDir = srcDir; this.dstDir = dstDir; this.dbName = name; this.srcDbPath = Paths.get(this.srcDir, name); this.dstDbPath = Paths.get(this.dstDir, name); - this.safe = safe; } @Override @@ -178,23 +175,15 @@ public boolean doConvert() throws Exception { FileUtils.createDirIfNotExists(dstDir); logger.info("Convert database {} start", this.dbName); - if (safe) { - convertLevelToRocks(); - compact(); - } else { - FileUtils.copyDir(Paths.get(srcDir), Paths.get(dstDir), dbName); - } + convertLevelToRocks(); + compact(); + boolean result = check() && createEngine(dstDbPath.toString()); long etime = System.currentTimeMillis(); if (result) { - if (safe) { - logger.info("Convert database {} successful end with {} key-value {} minutes", - this.dbName, this.srcDbKeyCount, (etime - startTime) / 1000.0 / 60); - } else { - logger.info("Convert database {} successful end {} minutes", - this.dbName, (etime - startTime) / 1000.0 / 60); - } + logger.info("Convert database {} successful end with {} key-value {} minutes", + this.dbName, this.srcDbKeyCount, (etime - startTime) / 1000.0 / 60); } else { logger.info("Convert database {} failure", this.dbName); @@ -234,8 +223,8 @@ private void batchInsert(RocksDB rocks, List keys, List values) * @throws Exception RocksDBException */ private void write(RocksDB rocks, org.rocksdb.WriteBatch batch) throws Exception { - try { - rocks.write(new org.rocksdb.WriteOptions(), batch); + try (org.rocksdb.WriteOptions writeOptions = new org.rocksdb.WriteOptions()) { + rocks.write(writeOptions, batch); } catch (RocksDBException e) { // retry if (maybeRetry(e)) { @@ -270,7 +259,8 @@ public void convertLevelToRocks() throws Exception { JniDBFactory.pushMemoryPool(1024 * 1024); try ( DB level = DBUtils.newLevelDb(srcDbPath); - RocksDB rocks = DBUtils.newRocksDbForBulkLoad(dstDbPath); + org.rocksdb.Options options = DBUtils.newDefaultRocksDbOptions(true, dbName); + RocksDB rocks = RocksDB.open(options, this.dstDbPath.toString()); DBIterator levelIterator = level.iterator( new org.iq80.leveldb.ReadOptions().fillCache(false))) { @@ -302,7 +292,8 @@ private void compact() throws RocksDBException { if (DBUtils.MARKET_PAIR_PRICE_TO_ORDER.equalsIgnoreCase(this.dbName)) { return; } - try (RocksDB rocks = DBUtils.newRocksDb(this.dstDbPath)) { + try (org.rocksdb.Options options = DBUtils.newDefaultRocksDbOptions(false, dbName); + RocksDB rocks = RocksDB.open(options, this.dstDbPath.toString())) { logger.info("compact database {} start", this.dbName); rocks.compactRange(); logger.info("compact database {} end", this.dbName); @@ -310,11 +301,9 @@ private void compact() throws RocksDBException { } private boolean check() throws RocksDBException { - if (!safe) { - return true; - } try ( - RocksDB rocks = DBUtils.newRocksDbReadOnly(this.dstDbPath); + org.rocksdb.Options options = DBUtils.newDefaultRocksDbOptions(false, dbName); + RocksDB rocks = RocksDB.openReadOnly(options, this.dstDbPath.toString()); org.rocksdb.ReadOptions r = new org.rocksdb.ReadOptions().setFillCache(false); RocksIterator rocksIterator = rocks.newIterator(r)) { diff --git a/plugins/src/main/java/org/tron/plugins/DbCopy.java b/plugins/src/main/java/common/org/tron/plugins/DbCopy.java similarity index 100% rename from plugins/src/main/java/org/tron/plugins/DbCopy.java rename to plugins/src/main/java/common/org/tron/plugins/DbCopy.java diff --git a/plugins/src/main/java/org/tron/plugins/DbLite.java b/plugins/src/main/java/common/org/tron/plugins/DbLite.java similarity index 99% rename from plugins/src/main/java/org/tron/plugins/DbLite.java rename to plugins/src/main/java/common/org/tron/plugins/DbLite.java index 732d4913021..3f8a6cb58c8 100644 --- a/plugins/src/main/java/org/tron/plugins/DbLite.java +++ b/plugins/src/main/java/common/org/tron/plugins/DbLite.java @@ -656,12 +656,13 @@ private boolean isLite(String databaseDir) throws RocksDBException, IOException private long getSecondBlock(String databaseDir) throws RocksDBException, IOException { long num = 0; DBInterface sourceBlockIndexDb = DbTool.getDB(databaseDir, BLOCK_INDEX_DB_NAME); - DBIterator iterator = sourceBlockIndexDb.iterator(); - iterator.seek(ByteArray.fromLong(1)); - if (iterator.hasNext()) { - num = Longs.fromByteArray(iterator.getKey()); + try (DBIterator iterator = sourceBlockIndexDb.iterator()) { + iterator.seek(ByteArray.fromLong(1)); + if (iterator.hasNext()) { + num = Longs.fromByteArray(iterator.getKey()); + } + return num; } - return num; } private DBInterface getCheckpointDb(String sourceDir) throws IOException, RocksDBException { diff --git a/plugins/src/main/java/org/tron/plugins/DbMove.java b/plugins/src/main/java/common/org/tron/plugins/DbMove.java similarity index 100% rename from plugins/src/main/java/org/tron/plugins/DbMove.java rename to plugins/src/main/java/common/org/tron/plugins/DbMove.java diff --git a/plugins/src/main/java/org/tron/plugins/DbRoot.java b/plugins/src/main/java/common/org/tron/plugins/DbRoot.java similarity index 84% rename from plugins/src/main/java/org/tron/plugins/DbRoot.java rename to plugins/src/main/java/common/org/tron/plugins/DbRoot.java index 7c33219e180..45854bbebdc 100644 --- a/plugins/src/main/java/org/tron/plugins/DbRoot.java +++ b/plugins/src/main/java/common/org/tron/plugins/DbRoot.java @@ -67,16 +67,24 @@ public Integer call() throws Exception { .errorText("Specify at least one exit database: --db dbName.")); return 404; } - List task = ProgressBar.wrap(dbs.stream(), "root task").parallel() - .map(this::calcMerkleRoot).collect(Collectors.toList()); - task.forEach(this::printInfo); - int code = (int) task.stream().filter(r -> r.code == 1).count(); - if (code > 0) { + try { + List task = ProgressBar.wrap(dbs.stream(), "root task").parallel() + .map(this::calcMerkleRoot).collect(Collectors.toList()); + task.forEach(this::printInfo); + int code = (int) task.stream().filter(r -> r.code == 1).count(); + if (code > 0) { + spec.commandLine().getErr().println(spec.commandLine().getColorScheme() + .errorText("There are some errors, please check toolkit.log for detail.")); + } + spec.commandLine().getOut().println("root task done."); + return code; + } catch (Exception e) { + logger.error("{}", e); spec.commandLine().getErr().println(spec.commandLine().getColorScheme() - .errorText("There are some errors, please check toolkit.log for detail.")); + .errorText(e.getMessage())); + spec.commandLine().usage(System.out); + return 1; } - spec.commandLine().getOut().println("root task done."); - return code; } private Ret calcMerkleRoot(String name) { diff --git a/plugins/src/main/java/org/tron/plugins/Toolkit.java b/plugins/src/main/java/common/org/tron/plugins/Toolkit.java similarity index 100% rename from plugins/src/main/java/org/tron/plugins/Toolkit.java rename to plugins/src/main/java/common/org/tron/plugins/Toolkit.java diff --git a/plugins/src/main/java/org/tron/plugins/utils/ByteArray.java b/plugins/src/main/java/common/org/tron/plugins/utils/ByteArray.java similarity index 100% rename from plugins/src/main/java/org/tron/plugins/utils/ByteArray.java rename to plugins/src/main/java/common/org/tron/plugins/utils/ByteArray.java diff --git a/plugins/src/main/java/org/tron/plugins/utils/CryptoUitls.java b/plugins/src/main/java/common/org/tron/plugins/utils/CryptoUitls.java similarity index 100% rename from plugins/src/main/java/org/tron/plugins/utils/CryptoUitls.java rename to plugins/src/main/java/common/org/tron/plugins/utils/CryptoUitls.java diff --git a/plugins/src/main/java/org/tron/plugins/utils/DBUtils.java b/plugins/src/main/java/common/org/tron/plugins/utils/DBUtils.java similarity index 72% rename from plugins/src/main/java/org/tron/plugins/utils/DBUtils.java rename to plugins/src/main/java/common/org/tron/plugins/utils/DBUtils.java index f8559d5dba8..6eb097cbec5 100644 --- a/plugins/src/main/java/org/tron/plugins/utils/DBUtils.java +++ b/plugins/src/main/java/common/org/tron/plugins/utils/DBUtils.java @@ -14,10 +14,9 @@ import org.rocksdb.BloomFilter; import org.rocksdb.ComparatorOptions; import org.rocksdb.Options; -import org.rocksdb.RocksDB; -import org.rocksdb.RocksDBException; -import org.tron.plugins.comparator.MarketOrderPriceComparatorForLevelDB; -import org.tron.plugins.comparator.MarketOrderPriceComparatorForRockDB; +import org.tron.common.arch.Arch; +import org.tron.common.utils.MarketOrderPriceComparatorForLevelDB; +import org.tron.common.utils.MarketOrderPriceComparatorForRocksDB; import org.tron.protos.Protocol; public class DBUtils { @@ -65,6 +64,7 @@ static Operator valueOf(byte b) { public static final String ROCKSDB = "ROCKSDB"; public static DB newLevelDb(Path db) throws IOException { + Arch.throwIfUnsupportedArm64Exception(LEVELDB); File file = db.toFile(); org.iq80.leveldb.Options dbOptions = newDefaultLevelDbOptions(); if (MARKET_PAIR_PRICE_TO_ORDER.equalsIgnoreCase(file.getName())) { @@ -86,7 +86,23 @@ public static org.iq80.leveldb.Options newDefaultLevelDbOptions() { return dbOptions; } - private static Options newDefaultRocksDbOptions(boolean forBulkLoad) { + /** + * Creates a new RocksDB Options. + * + *

CRITICAL: Must be closed after use to prevent native memory leaks. + * Use try-with-resources. + * + *

{@code
+   * try (Options options = newDefaultRocksDbOptions(false, name)) {
+   *     // do something
+   * }
+   * }
+ * + * @param forBulkLoad if true, optimizes for bulk loading + * @param name db name + * @return a new Options instance that must be closed + */ + public static Options newDefaultRocksDbOptions(boolean forBulkLoad, String name) { Options options = new Options(); options.setCreateIfMissing(true); options.setIncreaseParallelism(1); @@ -109,35 +125,10 @@ private static Options newDefaultRocksDbOptions(boolean forBulkLoad) { if (forBulkLoad) { options.prepareForBulkLoad(); } - return options; - } - - public static RocksDB newRocksDb(Path db) throws RocksDBException { - try (Options options = newDefaultRocksDbOptions(false)) { - if (MARKET_PAIR_PRICE_TO_ORDER.equalsIgnoreCase(db.getFileName().toString())) { - options.setComparator(new MarketOrderPriceComparatorForRockDB(new ComparatorOptions())); - } - return RocksDB.open(options, db.toString()); - } - } - - public static RocksDB newRocksDbForBulkLoad(Path db) throws RocksDBException { - try (Options options = newDefaultRocksDbOptions(true)) { - if (MARKET_PAIR_PRICE_TO_ORDER.equalsIgnoreCase(db.getFileName().toString())) { - options.setComparator(new MarketOrderPriceComparatorForRockDB(new ComparatorOptions())); - } - return RocksDB.open(options, db.toString()); - } - } - - - public static RocksDB newRocksDbReadOnly(Path db) throws RocksDBException { - try (Options options = newDefaultRocksDbOptions(false)) { - if (MARKET_PAIR_PRICE_TO_ORDER.equalsIgnoreCase(db.getFileName().toString())) { - options.setComparator(new MarketOrderPriceComparatorForRockDB(new ComparatorOptions())); - } - return RocksDB.openReadOnly(options, db.toString()); + if (MARKET_PAIR_PRICE_TO_ORDER.equalsIgnoreCase(name)) { + options.setComparator(new MarketOrderPriceComparatorForRocksDB(new ComparatorOptions())); } + return options; } public static String simpleDecode(byte[] bytes) { diff --git a/plugins/src/main/java/org/tron/plugins/utils/FileUtils.java b/plugins/src/main/java/common/org/tron/plugins/utils/FileUtils.java similarity index 100% rename from plugins/src/main/java/org/tron/plugins/utils/FileUtils.java rename to plugins/src/main/java/common/org/tron/plugins/utils/FileUtils.java diff --git a/plugins/src/main/java/common/org/tron/plugins/utils/MarketUtils.java b/plugins/src/main/java/common/org/tron/plugins/utils/MarketUtils.java new file mode 100644 index 00000000000..9bcfd5e71ce --- /dev/null +++ b/plugins/src/main/java/common/org/tron/plugins/utils/MarketUtils.java @@ -0,0 +1,68 @@ +package org.tron.plugins.utils; + +public class MarketUtils { + + public static final int TOKEN_ID_LENGTH = ByteArray + .fromString(Long.toString(Long.MAX_VALUE)).length; // 19 + + + + /** + * In order to avoid the difference between the data of same key stored and fetched by hashMap and + * levelDB, when creating the price key, we will find the GCD (Greatest Common Divisor) of + * sellTokenQuantity and buyTokenQuantity. + */ + public static byte[] createPairPriceKey(byte[] sellTokenId, byte[] buyTokenId, + long sellTokenQuantity, long buyTokenQuantity) { + + byte[] sellTokenQuantityBytes; + byte[] buyTokenQuantityBytes; + + // cal the GCD + long gcd = findGCD(sellTokenQuantity, buyTokenQuantity); + if (gcd == 0) { + sellTokenQuantityBytes = ByteArray.fromLong(sellTokenQuantity); + buyTokenQuantityBytes = ByteArray.fromLong(buyTokenQuantity); + } else { + sellTokenQuantityBytes = ByteArray.fromLong(sellTokenQuantity / gcd); + buyTokenQuantityBytes = ByteArray.fromLong(buyTokenQuantity / gcd); + } + + return doCreatePairPriceKey(sellTokenId, buyTokenId, + sellTokenQuantityBytes, buyTokenQuantityBytes); + } + + public static long findGCD(long number1, long number2) { + if (number1 == 0 || number2 == 0) { + return 0; + } + return calGCD(number1, number2); + } + + private static long calGCD(long number1, long number2) { + if (number2 == 0) { + return number1; + } + return calGCD(number2, number1 % number2); + } + + + private static byte[] doCreatePairPriceKey(byte[] sellTokenId, byte[] buyTokenId, + byte[] sellTokenQuantity, byte[] buyTokenQuantity) { + byte[] result = new byte[TOKEN_ID_LENGTH + TOKEN_ID_LENGTH + + sellTokenQuantity.length + buyTokenQuantity.length]; + + System.arraycopy(sellTokenId, 0, result, 0, sellTokenId.length); + System.arraycopy(buyTokenId, 0, result, TOKEN_ID_LENGTH, buyTokenId.length); + System.arraycopy(sellTokenQuantity, 0, result, + TOKEN_ID_LENGTH + TOKEN_ID_LENGTH, + sellTokenQuantity.length); + System.arraycopy(buyTokenQuantity, 0, result, + TOKEN_ID_LENGTH + TOKEN_ID_LENGTH + buyTokenQuantity.length, + buyTokenQuantity.length); + + return result; + } + + +} diff --git a/plugins/src/main/java/org/tron/plugins/utils/MerkleRoot.java b/plugins/src/main/java/common/org/tron/plugins/utils/MerkleRoot.java similarity index 100% rename from plugins/src/main/java/org/tron/plugins/utils/MerkleRoot.java rename to plugins/src/main/java/common/org/tron/plugins/utils/MerkleRoot.java diff --git a/plugins/src/main/java/org/tron/plugins/utils/Sha256Hash.java b/plugins/src/main/java/common/org/tron/plugins/utils/Sha256Hash.java similarity index 100% rename from plugins/src/main/java/org/tron/plugins/utils/Sha256Hash.java rename to plugins/src/main/java/common/org/tron/plugins/utils/Sha256Hash.java diff --git a/plugins/src/main/java/common/org/tron/plugins/utils/db/DBInterface.java b/plugins/src/main/java/common/org/tron/plugins/utils/db/DBInterface.java new file mode 100644 index 00000000000..13a195f9347 --- /dev/null +++ b/plugins/src/main/java/common/org/tron/plugins/utils/db/DBInterface.java @@ -0,0 +1,39 @@ +package org.tron.plugins.utils.db; + +import java.io.Closeable; +import java.io.IOException; + + +public interface DBInterface extends Closeable { + + byte[] get(byte[] key); + + void put(byte[] key, byte[] value); + + void delete(byte[] key); + + /** + * Returns an iterator over the database. + * + *

CRITICAL: The returned iterator holds native resources and MUST be closed + * after use to prevent memory leaks. It is strongly recommended to use a try-with-resources + * statement. + * + *

Example of correct usage: + *

{@code
+   * try (DBIterator iterator = db.iterator()) {
+   *  // do something
+   * }
+   * }
+ * + * @return a new database iterator that must be closed. + */ + DBIterator iterator(); + + long size() throws IOException; + + void close() throws IOException; + + String getName(); + +} diff --git a/plugins/src/main/java/org/tron/plugins/utils/db/DBIterator.java b/plugins/src/main/java/common/org/tron/plugins/utils/db/DBIterator.java similarity index 100% rename from plugins/src/main/java/org/tron/plugins/utils/db/DBIterator.java rename to plugins/src/main/java/common/org/tron/plugins/utils/db/DBIterator.java diff --git a/plugins/src/main/java/org/tron/plugins/utils/db/DbTool.java b/plugins/src/main/java/common/org/tron/plugins/utils/db/DbTool.java similarity index 85% rename from plugins/src/main/java/org/tron/plugins/utils/db/DbTool.java rename to plugins/src/main/java/common/org/tron/plugins/utils/db/DbTool.java index 429025e8f8b..127b8f97db5 100644 --- a/plugins/src/main/java/org/tron/plugins/utils/db/DbTool.java +++ b/plugins/src/main/java/common/org/tron/plugins/utils/db/DbTool.java @@ -18,8 +18,8 @@ public class DbTool { private static final String KEY_ENGINE = "ENGINE"; private static final String ENGINE_FILE = "engine.properties"; - private static final String FILE_SEPARATOR = File.separator; private static final String ROCKSDB = "ROCKSDB"; + private static final String LEVELDB = "LEVELDB"; private static final Map dbMap = Maps.newConcurrentMap(); @@ -162,8 +162,7 @@ public static void close() { } private static DbType getDbType(String sourceDir, String dbName) { - String engineFile = String.format("%s%s%s%s%s", sourceDir, FILE_SEPARATOR, - dbName, FILE_SEPARATOR, ENGINE_FILE); + String engineFile = Paths.get(sourceDir, dbName, ENGINE_FILE).toString(); if (!new File(engineFile).exists()) { return DbType.LevelDB; } @@ -175,13 +174,22 @@ private static DbType getDbType(String sourceDir, String dbName) { } } - private static LevelDBImpl openLevelDb(Path db, String name) throws IOException { - return new LevelDBImpl(DBUtils.newLevelDb(db), name); + public static LevelDBImpl openLevelDb(Path db, String name) throws IOException { + LevelDBImpl leveldb = new LevelDBImpl(DBUtils.newLevelDb(db), name); + tryInitEngineFile(db, LEVELDB); + return leveldb; } - private static RocksDBImpl openRocksDb(Path db, String name) throws RocksDBException { - return new RocksDBImpl(DBUtils.newRocksDb(db), name); + public static RocksDBImpl openRocksDb(Path db, String name) throws RocksDBException { + RocksDBImpl rocksdb = new RocksDBImpl(db, name); + tryInitEngineFile(db, ROCKSDB); + return rocksdb; } - + private static void tryInitEngineFile(Path db, String engine) { + String engineFile = Paths.get(db.toString(), ENGINE_FILE).toString(); + if (FileUtils.createFileIfNotExists(engineFile)) { + FileUtils.writeProperty(engineFile, KEY_ENGINE, engine); + } + } } diff --git a/plugins/src/main/java/org/tron/plugins/utils/db/LevelDBImpl.java b/plugins/src/main/java/common/org/tron/plugins/utils/db/LevelDBImpl.java similarity index 83% rename from plugins/src/main/java/org/tron/plugins/utils/db/LevelDBImpl.java rename to plugins/src/main/java/common/org/tron/plugins/utils/db/LevelDBImpl.java index 511f4dfd5b4..1c7f22eff1a 100644 --- a/plugins/src/main/java/org/tron/plugins/utils/db/LevelDBImpl.java +++ b/plugins/src/main/java/common/org/tron/plugins/utils/db/LevelDBImpl.java @@ -40,8 +40,11 @@ public DBIterator iterator() { } @Override - public long size() { - return Streams.stream(leveldb.iterator()).count(); + public long size() throws IOException { + try (DBIterator iterator = this.iterator()) { + iterator.seekToFirst(); + return Streams.stream(iterator).count(); + } } @Override diff --git a/plugins/src/main/java/org/tron/plugins/utils/db/LevelDBIterator.java b/plugins/src/main/java/common/org/tron/plugins/utils/db/LevelDBIterator.java similarity index 100% rename from plugins/src/main/java/org/tron/plugins/utils/db/LevelDBIterator.java rename to plugins/src/main/java/common/org/tron/plugins/utils/db/LevelDBIterator.java diff --git a/plugins/src/main/java/org/tron/plugins/utils/db/RockDBIterator.java b/plugins/src/main/java/common/org/tron/plugins/utils/db/RockDBIterator.java similarity index 76% rename from plugins/src/main/java/org/tron/plugins/utils/db/RockDBIterator.java rename to plugins/src/main/java/common/org/tron/plugins/utils/db/RockDBIterator.java index d3e17d9173f..17ecca4a4c1 100644 --- a/plugins/src/main/java/org/tron/plugins/utils/db/RockDBIterator.java +++ b/plugins/src/main/java/common/org/tron/plugins/utils/db/RockDBIterator.java @@ -2,14 +2,19 @@ import java.io.IOException; import java.util.Map; +import java.util.concurrent.atomic.AtomicBoolean; +import org.rocksdb.ReadOptions; import org.rocksdb.RocksIterator; public class RockDBIterator implements DBIterator { private final RocksIterator iterator; + private final ReadOptions readOptions; + private final AtomicBoolean closed = new AtomicBoolean(false); - public RockDBIterator(RocksIterator iterator) { + public RockDBIterator(RocksIterator iterator, ReadOptions readOptions) { this.iterator = iterator; + this.readOptions = readOptions; } @Override @@ -72,6 +77,9 @@ public byte[] setValue(byte[] value) { @Override public void close() throws IOException { - iterator.close(); + if (closed.compareAndSet(false, true)) { + readOptions.close(); + iterator.close(); + } } } diff --git a/plugins/src/main/java/common/org/tron/plugins/utils/db/RocksDBImpl.java b/plugins/src/main/java/common/org/tron/plugins/utils/db/RocksDBImpl.java new file mode 100644 index 00000000000..236d0a847b3 --- /dev/null +++ b/plugins/src/main/java/common/org/tron/plugins/utils/db/RocksDBImpl.java @@ -0,0 +1,97 @@ +package org.tron.plugins.utils.db; + +import com.google.common.collect.Streams; +import java.io.IOException; +import java.nio.file.Path; +import java.util.concurrent.atomic.AtomicBoolean; +import lombok.Getter; +import org.rocksdb.Options; +import org.rocksdb.ReadOptions; +import org.rocksdb.RocksDB; +import org.rocksdb.RocksDBException; +import org.tron.plugins.utils.DBUtils; + +public class RocksDBImpl implements DBInterface { + + private final RocksDB rocksDB; + + @Getter + private final String name; + private final AtomicBoolean closed = new AtomicBoolean(false); + private Options options = null; + + public RocksDBImpl(Path path, String name) throws RocksDBException { + try { + this.options = DBUtils.newDefaultRocksDbOptions(false, name); + this.name = name; + this.rocksDB = RocksDB.open(options, path.toString()); + } catch (RocksDBException e) { + if (this.options != null) { + this.options.close(); + } + throw e; + } + } + + @Override + public byte[] get(byte[] key) { + throwIfClosed(); + try { + return rocksDB.get(key); + } catch (RocksDBException e) { + throw new RuntimeException(name, e); + } + } + + @Override + public void put(byte[] key, byte[] value) { + throwIfClosed(); + try { + rocksDB.put(key, value); + } catch (RocksDBException e) { + throw new RuntimeException(name, e); + } + } + + @Override + public void delete(byte[] key) { + throwIfClosed(); + try { + rocksDB.delete(key); + } catch (RocksDBException e) { + throw new RuntimeException(name, e); + } + } + + @Override + public DBIterator iterator() { + throwIfClosed(); + ReadOptions readOptions = new ReadOptions().setFillCache(false); + return new RockDBIterator(rocksDB.newIterator(readOptions), readOptions); + } + + @Override + public long size() throws IOException { + throwIfClosed(); + try (DBIterator iterator = this.iterator()) { + iterator.seekToFirst(); + return Streams.stream(iterator).count(); + } + } + + @Override + public void close() throws IOException { + if (closed.compareAndSet(false, true)) { + if (this.options != null) { + this.options.close(); + } + rocksDB.close(); + } + } + + private void throwIfClosed() { + if (closed.get()) { + throw new IllegalStateException("db " + name + " has been closed"); + } + } +} diff --git a/plugins/src/main/java/org/tron/plugins/comparator/MarketOrderPriceComparatorForLevelDB.java b/plugins/src/main/java/org/tron/plugins/comparator/MarketOrderPriceComparatorForLevelDB.java deleted file mode 100644 index 0879f770e1f..00000000000 --- a/plugins/src/main/java/org/tron/plugins/comparator/MarketOrderPriceComparatorForLevelDB.java +++ /dev/null @@ -1,28 +0,0 @@ -package org.tron.plugins.comparator; - -import org.iq80.leveldb.DBComparator; -import org.tron.plugins.utils.MarketUtils; - -public class MarketOrderPriceComparatorForLevelDB implements DBComparator { - - @Override - public String name() { - return "MarketOrderPriceComparator"; - } - - @Override - public byte[] findShortestSeparator(byte[] start, byte[] limit) { - return new byte[0]; - } - - @Override - public byte[] findShortSuccessor(byte[] key) { - return new byte[0]; - } - - @Override - public int compare(byte[] o1, byte[] o2) { - return MarketUtils.comparePriceKey(o1, o2); - } - -} \ No newline at end of file diff --git a/plugins/src/main/java/org/tron/plugins/comparator/MarketOrderPriceComparatorForRockDB.java b/plugins/src/main/java/org/tron/plugins/comparator/MarketOrderPriceComparatorForRockDB.java deleted file mode 100644 index cd718bdd2d7..00000000000 --- a/plugins/src/main/java/org/tron/plugins/comparator/MarketOrderPriceComparatorForRockDB.java +++ /dev/null @@ -1,38 +0,0 @@ -package org.tron.plugins.comparator; - -import org.rocksdb.ComparatorOptions; -import org.rocksdb.DirectSlice; -import org.rocksdb.util.DirectBytewiseComparator; -import org.tron.plugins.utils.MarketUtils; - -public class MarketOrderPriceComparatorForRockDB extends DirectBytewiseComparator { - - public MarketOrderPriceComparatorForRockDB(final ComparatorOptions copt) { - super(copt); - } - - @Override - public String name() { - return "MarketOrderPriceComparator"; - } - - @Override - public int compare(final DirectSlice a, final DirectSlice b) { - return MarketUtils.comparePriceKey(convertDataToBytes(a), convertDataToBytes(b)); - } - - /** - * DirectSlice.data().array will throw UnsupportedOperationException. - * */ - public byte[] convertDataToBytes(DirectSlice directSlice) { - int capacity = directSlice.data().capacity(); - byte[] bytes = new byte[capacity]; - - for (int i = 0; i < capacity; i++) { - bytes[i] = directSlice.get(i); - } - - return bytes; - } - -} \ No newline at end of file diff --git a/plugins/src/main/java/org/tron/plugins/utils/db/DBInterface.java b/plugins/src/main/java/org/tron/plugins/utils/db/DBInterface.java deleted file mode 100644 index 513e021c83c..00000000000 --- a/plugins/src/main/java/org/tron/plugins/utils/db/DBInterface.java +++ /dev/null @@ -1,23 +0,0 @@ -package org.tron.plugins.utils.db; - -import java.io.Closeable; -import java.io.IOException; - - -public interface DBInterface extends Closeable { - - byte[] get(byte[] key); - - void put(byte[] key, byte[] value); - - void delete(byte[] key); - - DBIterator iterator(); - - long size(); - - void close() throws IOException; - - String getName(); - -} diff --git a/plugins/src/main/java/org/tron/plugins/utils/db/RocksDBImpl.java b/plugins/src/main/java/org/tron/plugins/utils/db/RocksDBImpl.java deleted file mode 100644 index 50957bbe61b..00000000000 --- a/plugins/src/main/java/org/tron/plugins/utils/db/RocksDBImpl.java +++ /dev/null @@ -1,69 +0,0 @@ -package org.tron.plugins.utils.db; - -import java.io.IOException; -import lombok.Getter; -import org.rocksdb.RocksDBException; -import org.rocksdb.RocksIterator; - -public class RocksDBImpl implements DBInterface { - - private org.rocksdb.RocksDB rocksDB; - - @Getter - private final String name; - - public RocksDBImpl(org.rocksdb.RocksDB rocksDB, String name) { - this.rocksDB = rocksDB; - this.name = name; - } - - @Override - public byte[] get(byte[] key) { - try { - return rocksDB.get(key); - } catch (RocksDBException e) { - e.printStackTrace(); - } - return null; - } - - @Override - public void put(byte[] key, byte[] value) { - try { - rocksDB.put(key, value); - } catch (RocksDBException e) { - e.printStackTrace(); - } - } - - @Override - public void delete(byte[] key) { - try { - rocksDB.delete(key); - } catch (RocksDBException e) { - e.printStackTrace(); - } - } - - @Override - public DBIterator iterator() { - return new RockDBIterator(rocksDB.newIterator( - new org.rocksdb.ReadOptions().setFillCache(false))); - } - - @Override - public long size() { - RocksIterator iterator = rocksDB.newIterator(); - long size = 0; - for (iterator.seekToFirst(); iterator.isValid(); iterator.next()) { - size++; - } - iterator.close(); - return size; - } - - @Override - public void close() throws IOException { - rocksDB.close(); - } -} diff --git a/plugins/src/main/java/org/tron/plugins/ArchiveManifest.java b/plugins/src/main/java/x86/org/tron/plugins/ArchiveManifest.java similarity index 98% rename from plugins/src/main/java/org/tron/plugins/ArchiveManifest.java rename to plugins/src/main/java/x86/org/tron/plugins/ArchiveManifest.java index 4d54df6d299..1d7a91027bf 100644 --- a/plugins/src/main/java/org/tron/plugins/ArchiveManifest.java +++ b/plugins/src/main/java/x86/org/tron/plugins/ArchiveManifest.java @@ -35,6 +35,7 @@ import org.iq80.leveldb.DB; import org.iq80.leveldb.Options; import org.iq80.leveldb.impl.Filename; +import org.tron.plugins.utils.DBUtils; import picocli.CommandLine; import picocli.CommandLine.Option; @@ -183,7 +184,7 @@ public boolean checkManifest(String dir) throws IOException { return false; } logger.info("CurrentName {}/{},size {} kb.", dir, currentName, current.length() / 1024); - if ("market_pair_price_to_order".equalsIgnoreCase(this.name)) { + if (DBUtils.MARKET_PAIR_PRICE_TO_ORDER.equalsIgnoreCase(this.name)) { logger.info("Db {} ignored.", this.name); return false; } diff --git a/plugins/src/main/java/org/tron/plugins/DbArchive.java b/plugins/src/main/java/x86/org/tron/plugins/DbArchive.java similarity index 98% rename from plugins/src/main/java/org/tron/plugins/DbArchive.java rename to plugins/src/main/java/x86/org/tron/plugins/DbArchive.java index e3032731ede..15bb281babf 100644 --- a/plugins/src/main/java/org/tron/plugins/DbArchive.java +++ b/plugins/src/main/java/x86/org/tron/plugins/DbArchive.java @@ -19,6 +19,7 @@ import org.iq80.leveldb.DB; import org.iq80.leveldb.Options; import org.iq80.leveldb.impl.Filename; +import org.tron.plugins.utils.DBUtils; import org.tron.plugins.utils.FileUtils; import picocli.CommandLine; import picocli.CommandLine.Option; @@ -163,7 +164,7 @@ public boolean checkManifest(String dir) throws IOException { return false; } logger.info("CurrentName {}/{},size {} kb.", dir, currentName, current.length() / 1024); - if ("market_pair_price_to_order".equalsIgnoreCase(this.name)) { + if (DBUtils.MARKET_PAIR_PRICE_TO_ORDER.equalsIgnoreCase(this.name)) { logger.info("Db {} ignored.", this.name); return false; } diff --git a/plugins/src/test/java/org/tron/plugins/DbCopyTest.java b/plugins/src/test/java/org/tron/plugins/DbCopyTest.java index 9e488a592aa..571fd8f5aa7 100644 --- a/plugins/src/test/java/org/tron/plugins/DbCopyTest.java +++ b/plugins/src/test/java/org/tron/plugins/DbCopyTest.java @@ -4,14 +4,25 @@ import java.util.UUID; import org.junit.Assert; import org.junit.Test; +import org.rocksdb.RocksDBException; +import org.tron.plugins.utils.db.DbTool; import picocli.CommandLine; public class DbCopyTest extends DbTest { @Test - public void testRun() { + public void testRunForLevelDB() throws RocksDBException, IOException { + init(DbTool.DbType.LevelDB); String[] args = new String[] { "db", "cp", INPUT_DIRECTORY, - genarateTmpDir()}; + generateTmpDir()}; + Assert.assertEquals(0, cli.execute(args)); + } + + @Test + public void testRunForRocksDB() throws RocksDBException, IOException { + init(DbTool.DbType.RocksDB); + String[] args = new String[] { "db", "cp", INPUT_DIRECTORY, + generateTmpDir()}; Assert.assertEquals(0, cli.execute(args)); } @@ -32,7 +43,7 @@ public void testNotExist() { @Test public void testEmpty() throws IOException { String[] args = new String[] {"db", "cp", temporaryFolder.newFolder().toString(), - genarateTmpDir()}; + generateTmpDir()}; Assert.assertEquals(0, cli.execute(args)); } @@ -46,7 +57,7 @@ public void testDestIsExist() throws IOException { @Test public void testSrcIsFile() throws IOException { String[] args = new String[] {"db", "cp", temporaryFolder.newFile().toString(), - genarateTmpDir()}; + generateTmpDir()}; Assert.assertEquals(403, cli.execute(args)); } diff --git a/plugins/src/test/java/org/tron/plugins/DbLiteTest.java b/plugins/src/test/java/org/tron/plugins/DbLiteTest.java index b4c66c9820f..80fd90cce81 100644 --- a/plugins/src/test/java/org/tron/plugins/DbLiteTest.java +++ b/plugins/src/test/java/org/tron/plugins/DbLiteTest.java @@ -7,10 +7,8 @@ import java.io.File; import java.io.IOException; import java.nio.file.Paths; -import java.util.concurrent.TimeUnit; import lombok.extern.slf4j.Slf4j; import org.junit.After; -import org.junit.ClassRule; import org.junit.Rule; import org.junit.rules.TemporaryFolder; import org.tron.api.WalletGrpc; @@ -21,6 +19,7 @@ import org.tron.common.crypto.ECKey; import org.tron.common.utils.FileUtil; import org.tron.common.utils.PublicMethod; +import org.tron.common.utils.TimeoutInterceptor; import org.tron.common.utils.Utils; import org.tron.core.config.DefaultConfig; import org.tron.core.config.args.Args; @@ -53,6 +52,7 @@ public void startApp() { Args.getInstance().getRpcPort()); channelFull = ManagedChannelBuilder.forTarget(fullNode) .usePlaintext() + .intercept(new TimeoutInterceptor(5000)) .build(); blockingStubFull = WalletGrpc.newBlockingStub(channelFull); } @@ -62,14 +62,15 @@ public void startApp() { */ public void shutdown() throws InterruptedException { if (channelFull != null) { - channelFull.shutdown().awaitTermination(5, TimeUnit.SECONDS); + channelFull.shutdownNow(); } context.close(); } - public void init() throws IOException { + public void init(String dbType) throws IOException { dbPath = folder.newFolder().toString(); - Args.setParam(new String[]{"-d", dbPath, "-w", "--p2p-disable", "true"}, + Args.setParam(new String[] { + "-d", dbPath, "-w", "--p2p-disable", "true", "--storage-db-engine", dbType}, "config-localtest.conf"); // allow account root Args.getInstance().setAllowAccountStateRoot(1); @@ -85,23 +86,22 @@ public void clear() { Args.clearParam(); } - void testTools(String dbType, int checkpointVersion) + public void testTools(String dbType, int checkpointVersion) throws InterruptedException, IOException { logger.info("dbType {}, checkpointVersion {}", dbType, checkpointVersion); dbPath = String.format("%s_%s_%d", dbPath, dbType, System.currentTimeMillis()); - init(); + init(dbType); final String[] argsForSnapshot = - new String[]{"-o", "split", "-t", "snapshot", "--fn-data-path", + new String[] {"-o", "split", "-t", "snapshot", "--fn-data-path", dbPath + File.separator + databaseDir, "--dataset-path", dbPath}; final String[] argsForHistory = - new String[]{"-o", "split", "-t", "history", "--fn-data-path", + new String[] {"-o", "split", "-t", "history", "--fn-data-path", dbPath + File.separator + databaseDir, "--dataset-path", dbPath}; final String[] argsForMerge = - new String[]{"-o", "merge", "--fn-data-path", dbPath + File.separator + databaseDir, + new String[] {"-o", "merge", "--fn-data-path", dbPath + File.separator + databaseDir, "--dataset-path", dbPath + File.separator + "history"}; - Args.getInstance().getStorage().setDbEngine(dbType); Args.getInstance().getStorage().setCheckpointVersion(checkpointVersion); DbLite.setRecentBlks(3); // start fullNode @@ -126,15 +126,15 @@ void testTools(String dbType, int checkpointVersion) File database = new File(Paths.get(dbPath, databaseDir).toString()); if (!database.renameTo(new File(Paths.get(dbPath, databaseDir + "_bak").toString()))) { throw new RuntimeException( - String.format("rename %s to %s failed", database.getPath(), - Paths.get(dbPath, databaseDir))); + String.format("rename %s to %s failed", database.getPath(), + Paths.get(dbPath, databaseDir))); } // change snapshot to the new database File snapshot = new File(Paths.get(dbPath, "snapshot").toString()); if (!snapshot.renameTo(new File(Paths.get(dbPath, databaseDir).toString()))) { throw new RuntimeException( - String.format("rename snapshot to %s failed", - Paths.get(dbPath, databaseDir))); + String.format("rename snapshot to %s failed", + Paths.get(dbPath, databaseDir))); } // start and validate the snapshot startApp(); @@ -161,7 +161,7 @@ private void generateSomeTransactions(int during) { String sunPri = getRandomPrivateKey(); byte[] sunAddress = PublicMethod.getFinalAddress(sunPri); PublicMethod.sendcoin(address, 1L, - sunAddress, sunPri, blockingStubFull); + sunAddress, sunPri, blockingStubFull); try { Thread.sleep(sleepOnce); } catch (InterruptedException e) { diff --git a/plugins/src/test/java/org/tron/plugins/DbMoveTest.java b/plugins/src/test/java/org/tron/plugins/DbMoveTest.java index c1bc6b470fc..5b25739f272 100644 --- a/plugins/src/test/java/org/tron/plugins/DbMoveTest.java +++ b/plugins/src/test/java/org/tron/plugins/DbMoveTest.java @@ -1,52 +1,41 @@ package org.tron.plugins; -import static org.iq80.leveldb.impl.Iq80DBFactory.factory; - import java.io.File; import java.io.IOException; import java.net.URL; import java.nio.file.Paths; -import java.util.UUID; import lombok.extern.slf4j.Slf4j; -import org.junit.AfterClass; +import org.junit.After; import org.junit.Assert; -import org.junit.BeforeClass; +import org.junit.Rule; import org.junit.Test; -import org.tron.plugins.utils.FileUtils; +import org.junit.rules.TemporaryFolder; +import org.rocksdb.RocksDBException; +import org.tron.plugins.utils.DBUtils; +import org.tron.plugins.utils.db.DbTool; import picocli.CommandLine; @Slf4j public class DbMoveTest { private static final String OUTPUT_DIRECTORY = "output-directory-toolkit"; - private static final String OUTPUT_DIRECTORY_DATABASE = - Paths.get(OUTPUT_DIRECTORY,"ori","database").toString(); - private static final String ENGINE = "ENGINE"; - private static final String LEVELDB = "LEVELDB"; - private static final String ACCOUNT = "account"; - private static final String TRANS = "trans"; - private static final String MARKET = "market_pair_price_to_order"; - private static final String ENGINE_FILE = "engine.properties"; + @Rule + public TemporaryFolder temporaryFolder = new TemporaryFolder(); - @BeforeClass - public static void init() throws IOException { - File file = new File(OUTPUT_DIRECTORY_DATABASE, ACCOUNT); - factory.open(file, ArchiveManifest.newDefaultLevelDbOptions()).close(); - FileUtils.writeProperty(file + File.separator + ENGINE_FILE, ENGINE, LEVELDB); + private static final String ACCOUNT = "account"; + private static final String TRANS = "trans"; - file = new File(OUTPUT_DIRECTORY_DATABASE, MARKET); - factory.open(file, ArchiveManifest.newDefaultLevelDbOptions()).close(); - FileUtils.writeProperty(file + File.separator + ENGINE_FILE, ENGINE, LEVELDB); - file = new File(OUTPUT_DIRECTORY_DATABASE, TRANS); - factory.open(file, ArchiveManifest.newDefaultLevelDbOptions()).close(); - FileUtils.writeProperty(file + File.separator + ENGINE_FILE, ENGINE, LEVELDB); + private void init(DbTool.DbType dbType, String path) throws IOException, RocksDBException { + DbTool.getDB(path, ACCOUNT, dbType).close(); + DbTool.getDB(path, DBUtils.MARKET_PAIR_PRICE_TO_ORDER, dbType).close(); + DbTool.getDB(path, TRANS, dbType).close(); } - @AfterClass - public static void destroy() { + @After + public void destroy() { deleteDir(new File(OUTPUT_DIRECTORY)); } @@ -74,9 +63,23 @@ private static String getConfig(String config) { } @Test - public void testMv() { + public void testMvForLevelDB() throws RocksDBException, IOException { + File database = temporaryFolder.newFolder("database"); + init(DbTool.DbType.LevelDB, Paths.get(database.getPath()).toString()); + String[] args = new String[] {"db", "mv", "-d", + database.getParent(), "-c", + getConfig("config.conf")}; + CommandLine cli = new CommandLine(new Toolkit()); + Assert.assertEquals(0, cli.execute(args)); + Assert.assertEquals(2, cli.execute(args)); + } + + @Test + public void testMvForRocksDB() throws RocksDBException, IOException { + File database = temporaryFolder.newFolder("database"); + init(DbTool.DbType.RocksDB, Paths.get(database.getPath()).toString()); String[] args = new String[] {"db", "mv", "-d", - Paths.get(OUTPUT_DIRECTORY,"ori").toString(), "-c", + database.getParent(), "-c", getConfig("config.conf")}; CommandLine cli = new CommandLine(new Toolkit()); Assert.assertEquals(0, cli.execute(args)); @@ -84,9 +87,10 @@ public void testMv() { } @Test - public void testDuplicate() { + public void testDuplicate() throws IOException { + File output = temporaryFolder.newFolder(); String[] args = new String[] {"db", "mv", "-d", - Paths.get(OUTPUT_DIRECTORY,"ori").toString(), "-c", + output.getPath(), "-c", getConfig("config-duplicate.conf")}; CommandLine cli = new CommandLine(new Toolkit()); Assert.assertEquals(2, cli.execute(args)); @@ -107,20 +111,19 @@ public void testDicNotExist() { } @Test - public void testConfNotExist() { + public void testConfNotExist() throws IOException { + File output = temporaryFolder.newFolder(); String[] args = new String[] {"db", "mv", "-d", - Paths.get(OUTPUT_DIRECTORY,"ori").toString(), "-c", + output.getPath(), "-c", "config.conf"}; CommandLine cli = new CommandLine(new Toolkit()); Assert.assertEquals(2, cli.execute(args)); } @Test - public void testEmpty() { - File file = new File(OUTPUT_DIRECTORY_DATABASE + File.separator + UUID.randomUUID()); - file.mkdirs(); - file.deleteOnExit(); - String[] args = new String[] {"db", "mv", "-d", file.toString(), "-c", + public void testEmpty() throws IOException { + File output = temporaryFolder.newFolder(); + String[] args = new String[] {"db", "mv", "-d", output.getPath(), "-c", getConfig("config.conf")}; CommandLine cli = new CommandLine(new Toolkit()); diff --git a/plugins/src/test/java/org/tron/plugins/DbRootTest.java b/plugins/src/test/java/org/tron/plugins/DbRootTest.java index b86688f77d5..9d032a43103 100644 --- a/plugins/src/test/java/org/tron/plugins/DbRootTest.java +++ b/plugins/src/test/java/org/tron/plugins/DbRootTest.java @@ -9,10 +9,8 @@ import org.junit.Test; import org.junit.rules.TemporaryFolder; import org.rocksdb.RocksDBException; -import org.tron.plugins.utils.DBUtils; import org.tron.plugins.utils.db.DBInterface; import org.tron.plugins.utils.db.DbTool; -import org.tron.plugins.utils.db.LevelDBImpl; import picocli.CommandLine; @Slf4j @@ -27,8 +25,18 @@ public class DbRootTest { private static final String EMPTY_DB = "empty"; private static final String ERROR_DB = "error"; + + @Test + public void testRootForLevelDB() throws RocksDBException, IOException { + testRoot(DbTool.DbType.LevelDB); + } + @Test - public void testRoot() throws IOException, RocksDBException { + public void testRootForRocksDB() throws RocksDBException, IOException { + testRoot(DbTool.DbType.RocksDB); + } + + public void testRoot(DbTool.DbType dbType) throws IOException, RocksDBException { File file = folder.newFolder(); @@ -36,8 +44,8 @@ public void testRoot() throws IOException, RocksDBException { Assert.assertTrue(database.mkdirs()); - try (DBInterface normal = DbTool.getDB(database.toString(), NORMAL_DB, DbTool.DbType.LevelDB); - DBInterface empty = DbTool.getDB(database.toString(), EMPTY_DB, DbTool.DbType.RocksDB)) { + try (DBInterface normal = DbTool.getDB(database.toString(), NORMAL_DB, dbType); + DBInterface empty = DbTool.getDB(database.toString(), EMPTY_DB, dbType)) { for (int i = 0; i < 10; i++) { normal.put(("" + i).getBytes(), (NORMAL_DB + "-" + i).getBytes()); } @@ -53,8 +61,7 @@ public void testRoot() throws IOException, RocksDBException { "--db", EMPTY_DB}; Assert.assertEquals(0, cli.execute(args)); - try (DBInterface errorDb = new LevelDBImpl( - DBUtils.newLevelDb(Paths.get(database.toString(), ERROR_DB)), ERROR_DB)) { + try (DBInterface errorDb = DbTool.getDB(database.toString(), ERROR_DB, dbType)) { for (int i = 0; i < 10; i++) { errorDb.put(("" + i).getBytes(), (ERROR_DB + "-" + i).getBytes()); } diff --git a/plugins/src/test/java/org/tron/plugins/DbTest.java b/plugins/src/test/java/org/tron/plugins/DbTest.java index 8605fa18d50..bbcc1a0bbf7 100644 --- a/plugins/src/test/java/org/tron/plugins/DbTest.java +++ b/plugins/src/test/java/org/tron/plugins/DbTest.java @@ -5,42 +5,43 @@ import java.nio.charset.StandardCharsets; import java.nio.file.Paths; import java.util.UUID; -import org.iq80.leveldb.DB; -import org.junit.Before; +import org.junit.Assert; import org.junit.Rule; import org.junit.rules.TemporaryFolder; +import org.rocksdb.RocksDBException; import org.tron.plugins.utils.ByteArray; import org.tron.plugins.utils.DBUtils; import org.tron.plugins.utils.MarketUtils; +import org.tron.plugins.utils.db.DBInterface; +import org.tron.plugins.utils.db.DbTool; import picocli.CommandLine; public class DbTest { - String INPUT_DIRECTORY; + public String INPUT_DIRECTORY; private static final String ACCOUNT = "account"; private static final String MARKET = DBUtils.MARKET_PAIR_PRICE_TO_ORDER; - CommandLine cli = new CommandLine(new Toolkit()); - String tmpDir = System.getProperty("java.io.tmpdir"); + public CommandLine cli = new CommandLine(new Toolkit()); @Rule public final TemporaryFolder temporaryFolder = new TemporaryFolder(); - @Before - public void init() throws IOException { + public void init(DbTool.DbType dbType) throws IOException, RocksDBException { INPUT_DIRECTORY = temporaryFolder.newFolder().toString(); - initDB(new File(INPUT_DIRECTORY, ACCOUNT)); - initDB(new File(INPUT_DIRECTORY, MARKET)); - initDB(new File(INPUT_DIRECTORY, DBUtils.CHECKPOINT_DB_V2)); + initDB(INPUT_DIRECTORY, ACCOUNT, dbType); + initDB(INPUT_DIRECTORY, MARKET, dbType); + initDB(INPUT_DIRECTORY, DBUtils.CHECKPOINT_DB_V2, dbType); } - private static void initDB(File file) throws IOException { - if (DBUtils.CHECKPOINT_DB_V2.equalsIgnoreCase(file.getName())) { - File dbFile = new File(file, DBUtils.CHECKPOINT_DB_V2); + private static void initDB(String sourceDir, String dbName, DbTool.DbType dbType) + throws IOException, RocksDBException { + if (DBUtils.CHECKPOINT_DB_V2.equalsIgnoreCase(dbName)) { + File dbFile = new File(sourceDir, DBUtils.CHECKPOINT_DB_V2); if (dbFile.mkdirs()) { for (int i = 0; i < 3; i++) { - try (DB db = DBUtils.newLevelDb(Paths.get(dbFile.getPath(), - System.currentTimeMillis() + ""))) { + try (DBInterface db = DbTool.getDB(dbFile.getPath(), + System.currentTimeMillis() + "", dbType)) { for (int j = 0; j < 100; j++) { byte[] bytes = UUID.randomUUID().toString().getBytes(); db.put(bytes, bytes); @@ -50,8 +51,8 @@ private static void initDB(File file) throws IOException { } return; } - try (DB db = DBUtils.newLevelDb(file.toPath())) { - if (MARKET.equalsIgnoreCase(file.getName())) { + try (DBInterface db = DbTool.getDB(sourceDir, dbName, dbType)) { + if (MARKET.equalsIgnoreCase(dbName)) { byte[] sellTokenID1 = ByteArray.fromString("100"); byte[] buyTokenID1 = ByteArray.fromString("200"); byte[] pairPriceKey1 = MarketUtils.createPairPriceKey( @@ -73,26 +74,29 @@ private static void initDB(File file) throws IOException { 2003L ); - //Use out-of-order insertion,key in store should be 1,2,3 db.put(pairPriceKey1, "1".getBytes(StandardCharsets.UTF_8)); db.put(pairPriceKey2, "2".getBytes(StandardCharsets.UTF_8)); db.put(pairPriceKey3, "3".getBytes(StandardCharsets.UTF_8)); + Assert.assertEquals(3, db.size()); } else { for (int i = 0; i < 100; i++) { byte[] bytes = UUID.randomUUID().toString().getBytes(); db.put(bytes, bytes); } + Assert.assertEquals(100, db.size()); } } } /** * Generate a not-exist temporary directory path. + * * @return temporary path */ - public String genarateTmpDir() { - File dir = Paths.get(tmpDir, UUID.randomUUID().toString()).toFile(); + public String generateTmpDir() throws IOException { + File dir = Paths.get(temporaryFolder.newFolder().toString(), UUID.randomUUID().toString()) + .toFile(); dir.deleteOnExit(); return dir.getPath(); } diff --git a/plugins/src/test/java/org/tron/plugins/ArchiveManifestTest.java b/plugins/src/test/java/org/tron/plugins/leveldb/ArchiveManifestTest.java similarity index 69% rename from plugins/src/test/java/org/tron/plugins/ArchiveManifestTest.java rename to plugins/src/test/java/org/tron/plugins/leveldb/ArchiveManifestTest.java index 4fd9e537d05..f5880d82e39 100644 --- a/plugins/src/test/java/org/tron/plugins/ArchiveManifestTest.java +++ b/plugins/src/test/java/org/tron/plugins/leveldb/ArchiveManifestTest.java @@ -1,6 +1,4 @@ -package org.tron.plugins; - -import static org.iq80.leveldb.impl.Iq80DBFactory.factory; +package org.tron.plugins.leveldb; import java.io.BufferedReader; import java.io.BufferedWriter; @@ -14,47 +12,41 @@ import java.nio.charset.StandardCharsets; import java.util.Properties; import java.util.UUID; - import lombok.extern.slf4j.Slf4j; -import org.junit.AfterClass; import org.junit.Assert; import org.junit.BeforeClass; +import org.junit.ClassRule; import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.rocksdb.RocksDBException; +import org.tron.plugins.ArchiveManifest; +import org.tron.plugins.utils.DBUtils; +import org.tron.plugins.utils.db.DbTool; @Slf4j public class ArchiveManifestTest { - private static final String OUTPUT_DIRECTORY = "output-directory/database/archiveManifest"; + @ClassRule + public static final TemporaryFolder temporaryFolder = new TemporaryFolder(); + + private static String OUTPUT_DIRECTORY; - private static final String ENGINE = "ENGINE"; - private static final String LEVELDB = "LEVELDB"; - private static final String ROCKSDB = "ROCKSDB"; private static final String ACCOUNT = "account"; private static final String ACCOUNT_ROCKSDB = "account-rocksdb"; - private static final String MARKET = "market_pair_price_to_order"; - private static final String ENGINE_FILE = "engine.properties"; - @BeforeClass - public static void init() throws IOException { - File file = new File(OUTPUT_DIRECTORY,ACCOUNT); - factory.open(file,ArchiveManifest.newDefaultLevelDbOptions()).close(); - writeProperty(file.toString() + File.separator + ENGINE_FILE,ENGINE,LEVELDB); - - file = new File(OUTPUT_DIRECTORY,MARKET); - factory.open(file,ArchiveManifest.newDefaultLevelDbOptions()).close(); - writeProperty(file.toString() + File.separator + ENGINE_FILE,ENGINE,LEVELDB); + public static void init() throws IOException, RocksDBException { + OUTPUT_DIRECTORY = temporaryFolder.newFolder("database").toString(); + File file = new File(OUTPUT_DIRECTORY, ACCOUNT); + DbTool.openLevelDb(file.toPath(),ACCOUNT).close(); - file = new File(OUTPUT_DIRECTORY,ACCOUNT_ROCKSDB); - factory.open(file,ArchiveManifest.newDefaultLevelDbOptions()).close(); - writeProperty(file.toString() + File.separator + ENGINE_FILE,ENGINE,ROCKSDB); + file = new File(OUTPUT_DIRECTORY, DBUtils.MARKET_PAIR_PRICE_TO_ORDER); + DbTool.openLevelDb(file.toPath(), DBUtils.MARKET_PAIR_PRICE_TO_ORDER).close(); - } + file = new File(OUTPUT_DIRECTORY, ACCOUNT_ROCKSDB); + DbTool.openRocksDb(file.toPath(), ACCOUNT_ROCKSDB).close(); - @AfterClass - public static void destroy() { - deleteDir(new File(OUTPUT_DIRECTORY)); } @Test diff --git a/plugins/src/test/java/org/tron/plugins/DbArchiveTest.java b/plugins/src/test/java/org/tron/plugins/leveldb/DbArchiveTest.java similarity index 71% rename from plugins/src/test/java/org/tron/plugins/DbArchiveTest.java rename to plugins/src/test/java/org/tron/plugins/leveldb/DbArchiveTest.java index 10bed418764..69dca01e4f8 100644 --- a/plugins/src/test/java/org/tron/plugins/DbArchiveTest.java +++ b/plugins/src/test/java/org/tron/plugins/leveldb/DbArchiveTest.java @@ -1,6 +1,4 @@ -package org.tron.plugins; - -import static org.iq80.leveldb.impl.Iq80DBFactory.factory; +package org.tron.plugins.leveldb; import java.io.BufferedReader; import java.io.BufferedWriter; @@ -12,48 +10,43 @@ import java.io.OutputStream; import java.io.OutputStreamWriter; import java.nio.charset.StandardCharsets; -import java.nio.file.Paths; import java.util.Properties; import java.util.UUID; import lombok.extern.slf4j.Slf4j; -import org.junit.AfterClass; import org.junit.Assert; import org.junit.BeforeClass; +import org.junit.ClassRule; import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.rocksdb.RocksDBException; +import org.tron.plugins.Toolkit; +import org.tron.plugins.utils.DBUtils; +import org.tron.plugins.utils.db.DbTool; import picocli.CommandLine; @Slf4j public class DbArchiveTest { - private static final String OUTPUT_DIRECTORY = "output-directory/database/dbArchive"; + @ClassRule + public static final TemporaryFolder temporaryFolder = new TemporaryFolder(); + + private static String OUTPUT_DIRECTORY; - private static final String ENGINE = "ENGINE"; - private static final String LEVELDB = "LEVELDB"; - private static final String ROCKSDB = "ROCKSDB"; private static final String ACCOUNT = "account"; private static final String ACCOUNT_ROCKSDB = "account-rocksdb"; - private static final String MARKET = "market_pair_price_to_order"; - private static final String ENGINE_FILE = "engine.properties"; @BeforeClass - public static void init() throws IOException { - File file = new File(OUTPUT_DIRECTORY,ACCOUNT); - factory.open(file,ArchiveManifest.newDefaultLevelDbOptions()).close(); - writeProperty(file.toString() + File.separator + ENGINE_FILE,ENGINE,LEVELDB); + public static void init() throws IOException, RocksDBException { + OUTPUT_DIRECTORY = temporaryFolder.newFolder("database").toString(); + File file = new File(OUTPUT_DIRECTORY, ACCOUNT); + DbTool.openLevelDb(file.toPath(),ACCOUNT).close(); - file = new File(OUTPUT_DIRECTORY,MARKET); - factory.open(file,ArchiveManifest.newDefaultLevelDbOptions()).close(); - writeProperty(file.toString() + File.separator + ENGINE_FILE,ENGINE,LEVELDB); + file = new File(OUTPUT_DIRECTORY, DBUtils.MARKET_PAIR_PRICE_TO_ORDER); + DbTool.openLevelDb(file.toPath(), DBUtils.MARKET_PAIR_PRICE_TO_ORDER).close(); - file = new File(OUTPUT_DIRECTORY,ACCOUNT_ROCKSDB); - factory.open(file,ArchiveManifest.newDefaultLevelDbOptions()).close(); - writeProperty(file.toString() + File.separator + ENGINE_FILE,ENGINE,ROCKSDB); - - } + file = new File(OUTPUT_DIRECTORY, ACCOUNT_ROCKSDB); + DbTool.openRocksDb(file.toPath(), ACCOUNT_ROCKSDB).close(); - @AfterClass - public static void destroy() { - deleteDir(new File(OUTPUT_DIRECTORY)); } @Test diff --git a/plugins/src/test/java/org/tron/plugins/DbConvertTest.java b/plugins/src/test/java/org/tron/plugins/leveldb/DbConvertTest.java similarity index 76% rename from plugins/src/test/java/org/tron/plugins/DbConvertTest.java rename to plugins/src/test/java/org/tron/plugins/leveldb/DbConvertTest.java index 150e47c9f65..d24604f0a0b 100644 --- a/plugins/src/test/java/org/tron/plugins/DbConvertTest.java +++ b/plugins/src/test/java/org/tron/plugins/leveldb/DbConvertTest.java @@ -1,27 +1,25 @@ -package org.tron.plugins; +package org.tron.plugins.leveldb; import java.io.IOException; import java.util.UUID; import org.junit.Assert; import org.junit.Test; +import org.rocksdb.RocksDBException; +import org.tron.plugins.DbTest; +import org.tron.plugins.Toolkit; +import org.tron.plugins.utils.db.DbTool; import picocli.CommandLine; public class DbConvertTest extends DbTest { @Test - public void testRun() throws IOException { + public void testRun() throws IOException, RocksDBException { + init(DbTool.DbType.LevelDB); String[] args = new String[] { "db", "convert", INPUT_DIRECTORY, temporaryFolder.newFolder().toString() }; Assert.assertEquals(0, cli.execute(args)); } - @Test - public void testRunWithSafe() throws IOException { - String[] args = new String[] { "db", "convert", INPUT_DIRECTORY, - temporaryFolder.newFolder().toString(),"--safe" }; - Assert.assertEquals(0, cli.execute(args)); - } - @Test public void testHelp() { String[] args = new String[] {"db", "convert", "-h"}; diff --git a/plugins/src/test/java/org/tron/plugins/DbLiteLevelDbTest.java b/plugins/src/test/java/org/tron/plugins/leveldb/DbLiteLevelDbTest.java similarity index 76% rename from plugins/src/test/java/org/tron/plugins/DbLiteLevelDbTest.java rename to plugins/src/test/java/org/tron/plugins/leveldb/DbLiteLevelDbTest.java index 792f93ad197..7666806e2b5 100644 --- a/plugins/src/test/java/org/tron/plugins/DbLiteLevelDbTest.java +++ b/plugins/src/test/java/org/tron/plugins/leveldb/DbLiteLevelDbTest.java @@ -1,7 +1,8 @@ -package org.tron.plugins; +package org.tron.plugins.leveldb; import java.io.IOException; import org.junit.Test; +import org.tron.plugins.DbLiteTest; public class DbLiteLevelDbTest extends DbLiteTest { diff --git a/plugins/src/test/java/org/tron/plugins/DbLiteLevelDbV2Test.java b/plugins/src/test/java/org/tron/plugins/leveldb/DbLiteLevelDbV2Test.java similarity index 76% rename from plugins/src/test/java/org/tron/plugins/DbLiteLevelDbV2Test.java rename to plugins/src/test/java/org/tron/plugins/leveldb/DbLiteLevelDbV2Test.java index ae48e1d66e9..de32ae29c7c 100644 --- a/plugins/src/test/java/org/tron/plugins/DbLiteLevelDbV2Test.java +++ b/plugins/src/test/java/org/tron/plugins/leveldb/DbLiteLevelDbV2Test.java @@ -1,7 +1,8 @@ -package org.tron.plugins; +package org.tron.plugins.leveldb; import java.io.IOException; import org.junit.Test; +import org.tron.plugins.DbLiteTest; public class DbLiteLevelDbV2Test extends DbLiteTest { diff --git a/plugins/src/test/java/org/tron/plugins/DbLiteRocksDbTest.java b/plugins/src/test/java/org/tron/plugins/rocksdb/DbLiteRocksDbTest.java similarity index 76% rename from plugins/src/test/java/org/tron/plugins/DbLiteRocksDbTest.java rename to plugins/src/test/java/org/tron/plugins/rocksdb/DbLiteRocksDbTest.java index e6910b1103a..2f9c92f9679 100644 --- a/plugins/src/test/java/org/tron/plugins/DbLiteRocksDbTest.java +++ b/plugins/src/test/java/org/tron/plugins/rocksdb/DbLiteRocksDbTest.java @@ -1,7 +1,8 @@ -package org.tron.plugins; +package org.tron.plugins.rocksdb; import java.io.IOException; import org.junit.Test; +import org.tron.plugins.DbLiteTest; public class DbLiteRocksDbTest extends DbLiteTest { diff --git a/plugins/src/test/java/org/tron/plugins/rocksdb/DbLiteRocksDbV2Test.java b/plugins/src/test/java/org/tron/plugins/rocksdb/DbLiteRocksDbV2Test.java new file mode 100644 index 00000000000..ab1067fefc3 --- /dev/null +++ b/plugins/src/test/java/org/tron/plugins/rocksdb/DbLiteRocksDbV2Test.java @@ -0,0 +1,13 @@ +package org.tron.plugins.rocksdb; + +import java.io.IOException; +import org.junit.Test; +import org.tron.plugins.DbLiteTest; + +public class DbLiteRocksDbV2Test extends DbLiteTest { + + @Test + public void testToolsWithRocksDB() throws InterruptedException, IOException { + testTools("ROCKSDB", 2); + } +} diff --git a/plugins/src/test/resources/config.conf b/plugins/src/test/resources/config.conf index 2bfca7dbdd7..77d15d521eb 100644 --- a/plugins/src/test/resources/config.conf +++ b/plugins/src/test/resources/config.conf @@ -1,11 +1,5 @@ storage { - # Directory for storing persistent data - db.engine = "LEVELDB", - db.sync = false, - db.directory = "database", - index.directory = "index", - transHistory.switch = "on", properties = [ { name = "account", diff --git a/protocol/build.gradle b/protocol/build.gradle index 535fac43a65..789d27b6360 100644 --- a/protocol/build.gradle +++ b/protocol/build.gradle @@ -1,7 +1,7 @@ apply plugin: 'com.google.protobuf' -def protobufVersion = '3.25.5' -def grpcVersion = '1.60.0' +def protobufVersion = '3.25.8' +def grpcVersion = '1.75.0' dependencies { api group: 'com.google.protobuf', name: 'protobuf-java', version: protobufVersion @@ -13,6 +13,7 @@ dependencies { api group: 'io.grpc', name: 'grpc-netty', version: grpcVersion api group: 'io.grpc', name: 'grpc-protobuf', version: grpcVersion api group: 'io.grpc', name: 'grpc-stub', version: grpcVersion + api group: 'io.grpc', name: 'grpc-core', version: grpcVersion api group: 'io.grpc', name: 'grpc-services', version: grpcVersion // end google grpc @@ -45,7 +46,7 @@ protobuf { plugins { grpc { - artifact = "io.grpc:protoc-gen-grpc-java:${grpcVersion}" + artifact = "io.grpc:protoc-gen-grpc-java:${rootProject.archInfo.requires.ProtocGenVersion}" } } generateProtoTasks { diff --git a/protocol/src/main/protos/api/api.proto b/protocol/src/main/protos/api/api.proto index 2505fa48d6f..a67113cb606 100644 --- a/protocol/src/main/protos/api/api.proto +++ b/protocol/src/main/protos/api/api.proto @@ -499,6 +499,8 @@ service Wallet { }; }; + rpc GetPaginatedNowWitnessList (PaginatedMessage) returns (WitnessList) { + }; rpc GetDelegatedResource (DelegatedResourceMessage) returns (DelegatedResourceList) { }; @@ -808,6 +810,10 @@ service WalletSolidity { } }; }; + + rpc GetPaginatedNowWitnessList (PaginatedMessage) returns (WitnessList) { + }; + rpc GetAssetIssueList (EmptyMessage) returns (AssetIssueList) { option (google.api.http) = { post: "/walletsolidity/getassetissuelist" diff --git a/protocol/src/main/protos/core/Tron.proto b/protocol/src/main/protos/core/Tron.proto index 2ffefbf9f3e..2b104b86d34 100644 --- a/protocol/src/main/protos/core/Tron.proto +++ b/protocol/src/main/protos/core/Tron.proto @@ -132,7 +132,7 @@ message ChainParameters { message Account { /* frozen balance */ message Frozen { - int64 frozen_balance = 1; // the frozen trx balance + int64 frozen_balance = 1; // the frozen trx or asset balance int64 expire_time = 2; // the expire time } // account nick name @@ -601,7 +601,7 @@ enum ReasonCode { CONNECT_FAIL = 0x21; TOO_MANY_PEERS_WITH_SAME_IP = 0x22; LIGHT_NODE_SYNC_FAIL = 0x23; - BELOW_THAN_ME = 0X24; + BELOW_THAN_ME = 0x24; NOT_WITNESS = 0x25; NO_SUCH_MESSAGE = 0x26; UNKNOWN = 0xFF; diff --git a/protocol/src/main/protos/core/contract/asset_issue_contract.proto b/protocol/src/main/protos/core/contract/asset_issue_contract.proto index eb86b170219..9e8ff463d52 100644 --- a/protocol/src/main/protos/core/contract/asset_issue_contract.proto +++ b/protocol/src/main/protos/core/contract/asset_issue_contract.proto @@ -10,7 +10,7 @@ message AssetIssueContract { string id = 41; message FrozenSupply { - int64 frozen_amount = 1; + int64 frozen_amount = 1; // asset amount int64 frozen_days = 2; } bytes owner_address = 1; @@ -18,7 +18,7 @@ message AssetIssueContract { bytes abbr = 3; int64 total_supply = 4; repeated FrozenSupply frozen_supply = 5; - int32 trx_num = 6; + int32 trx_num = 6; // The fields trx_num and num define the exchange rate: num tokens can be purchased with trx_num TRX. This avoids using decimals. int32 precision = 7; int32 num = 8; int64 start_time = 9; diff --git a/settings.gradle b/settings.gradle index eb304444378..af32bfca702 100644 --- a/settings.gradle +++ b/settings.gradle @@ -8,4 +8,5 @@ include 'common' include 'example:actuator-example' include 'crypto' include 'plugins' +include 'platform' diff --git a/start.sh.simple b/start.sh.simple new file mode 100644 index 00000000000..52548dea62b --- /dev/null +++ b/start.sh.simple @@ -0,0 +1,192 @@ +#!/bin/bash +############################################################################# +# +# GNU LESSER GENERAL PUBLIC LICENSE +# Version 3, 29 June 2007 +# +# Copyright (C) [2007] [TRON Foundation], Inc. +# Everyone is permitted to copy and distribute verbatim copies +# of this license document, but changing it is not allowed. +# +# +# This version of the GNU Lesser General Public License incorporates +# the terms and conditions of version 3 of the GNU General Public +# License, supplemented by the additional permissions listed below. +# +# You can find java-tron at https://github.com/tronprotocol/java-tron/ +# +############################################################################## +# TRON Full Node Management Simple Script +# +# NOTE: This is a simple and concise script to start and stop the java-tron full node, +# designed for developers to quickly get started and learn. +# It may not be suitable for production environments. +# +# Usage: +# sh start.sh # Start the java-tron FullNode +# sh start.sh -s # Stop the java-tron FullNode +# sh start.sh [options] # Start with additional java-tron options,such as: -c config.conf -d /path_to_data, etc. +# +############################################################################## + + +# adjust JVM start +# Set the maximum heap size to 9G, adjust as needed +VM_XMX="9G" +# adjust JVM end + +FULL_NODE_JAR="FullNode.jar" +FULL_START_OPT=() +PID="" +MAX_STOP_TIME=60 +JAVACMD="" +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +NC='\033[0m' + +log() { + local level="$1"; shift + local timestamp color="" + timestamp=$(date '+%Y-%m-%d %H:%M:%S') + case "$level" in + INFO) color="$GREEN" ;; + WARN) color="$YELLOW" ;; + ERROR) color="$RED" ;; + esac + printf "%b[%s] [%s]:%b %s\n" "$color" "$timestamp" "$level" "$NC" "$*" | tee -a "${SCRIPT_DIR}/start.log" +} + +info() { log INFO "$@"; } +warn() { log WARN "$@"; } +error() { log ERROR "$@"; } +die() { error "$@"; exit 1; } + +ulimit -n 65535 || warn "Failed to set ulimit -n 65535" + +findJava() { + if [ -n "${JAVA_HOME:-}" ]; then + if [ -x "$JAVA_HOME/jre/sh/java" ]; then + JAVACMD="$JAVA_HOME/jre/sh/java" + else + JAVACMD="$JAVA_HOME/bin/java" + fi + [ -x "$JAVACMD" ] || die "JAVA_HOME is invalid: $JAVA_HOME" + else + JAVACMD="java" + which java >/dev/null 2>&1 || die "JAVA_HOME not set and no 'java' in PATH" + fi + "$JAVACMD" -version > /dev/null 2>&1 || die "Java command not working" +} + +checkPid() { + # shellcheck disable=SC2009 + PID=$(ps -ef |grep $FULL_NODE_JAR |grep -v grep |awk '{print $2}') +} + + +stopService() { + checkPid + + if ! kill -0 "$PID" 2>/dev/null; then + info "java-tron is not running." + return 0 + fi + info "Stopping java-tron service (PID: $PID)" + + local count=1 + + while [ -n "$PID" ] && [ $count -le $MAX_STOP_TIME ]; do + kill -TERM "$PID" 2>/dev/null && info "Sent SIGTERM to java-tron (PID: $PID), attempt $count" + sleep 1 + checkPid + count=$((count + 1)) + done + + if [ -n "$PID" ]; then + warn "Forcing kill java-tron (PID: $PID) after $MAX_STOP_TIME seconds" + kill -KILL "$PID" 2>/dev/null + sleep 1 + checkPid + fi + + if [ -n "$PID" ]; then + die "Failed to stop the service (PID: $PID)" + else + info "java-tron stopped" + wait_with_info 2 "Cleaning up..." + fi +} + +startService() { + if [ -n "${FULL_START_OPT[*]}" ]; then + info "Starting java-tron service with options: ${FULL_START_OPT[*]}" + fi + if [ ! -f "$FULL_NODE_JAR" ]; then + die "$FULL_NODE_JAR not found in path $SCRIPT_DIR." + fi + + nohup "$JAVACMD" \ + -Xmx"$VM_XMX" \ + -XX:+UseZGC \ + -Xlog:gc,gc+heap:file=gc.log:time,tags,level:filecount=10,filesize=100M \ + -XX:ReservedCodeCacheSize=256m \ + -XX:+UseCodeCacheFlushing \ + -XX:MetaspaceSize=256m \ + -XX:MaxMetaspaceSize=512m \ + -XX:MaxDirectMemorySize=1g \ + -XX:+HeapDumpOnOutOfMemoryError \ + -jar "$FULL_NODE_JAR" "${FULL_START_OPT[@]}" \ + >> start.log 2>&1 & + + + info "Waiting for the service to start..." + wait_with_info 5 "Starting..." + + checkPid + + if [ -n "$PID" ]; then + info "Started java-tron with PID $PID on $HOSTNAME." + else + die "Failed to start java-tron, see start.log or logs/tron.log for details." + fi +} + +wait_with_info() { + local seconds=$1 + local message=$2 + for i in $(seq "$seconds" -1 1); do + info "$message wait ($i) s" + sleep 1 + done +} + + +start() { + checkPid + if [ -n "$PID" ]; then + info "java-tron is already running (PID: $PID), to stop the service: sh start.sh -s" + return + fi + findJava + startService +} + +while [ -n "$1" ]; do + case "$1" in + -s) + stopService + exit 0 + ;; + *) + FULL_START_OPT+=("$@") + break + ;; + esac +done + +start + +exit 0 \ No newline at end of file