Skip to content

Latest commit

 

History

History
224 lines (200 loc) · 10.1 KB

README.md

File metadata and controls

224 lines (200 loc) · 10.1 KB
Raw

Run MongoDB Atlas locally for testing

build codecov

Prerequisite

  • Java 8+

  • Docker

  • Chart shows local and remote docker support for replicaSetNumber

    replicaSetNumber local docker host local docker host running tests from inside a container with mapping the Docker socket remote docker daemon availability of an arbiter node
    1 + + + -
    from 2 to 7 (including) only if adding either host.docker.internal (your Docker version should support it) or dockerhost to the OS host file. See Supported features for details + + +

Tip: A single node replica set is the fastest among others. That is the default mode for MongoDbReplicaSet. However, to use only it, consider the Testcontainers MongoDB module on GitHub

Getting it

  • Gradle:
dependencies {
    testCompile("com.github.silaev:mongodb-replica-set:${LATEST_RELEASE}")
}
  • Maven:
<dependencies>
    <dependency>
        <groupId>com.github.silaev</groupId>
        <artifactId>mongodb-replica-set</artifactId>
        <version>${LATEST_RELEASE}</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Replace ${LATEST_RELEASE} with the Latest Version Number If you cannot find a release on Maven, please, use Jitpack

Run on Apple silicon

Use digests for linux/arm64/v8 and Docker Desktop for Apple silicon supporting host.docker.internal (should be in the OS host file). Examples: 1.

MongoDbReplicaSet.builder()
    .mongoDockerImageName("mongo@sha256:8a823923d80e819e21ee6c179eabf42460b6b7d8ac3dd5f35b59419ae5413640")
    .useHostDockerInternal(true)
    .build()`

2 ./gradlew clean build -DmongoReplicaSetProperties.mongoDockerImageName=mongo@sha256:8a823923d80e819e21ee6c179eabf42460b6b7d8ac3dd5f35b59419ae5413640 -DmongoReplicaSetProperties.useHostDockerInternal=true

MongoDB versions that MongoDbReplicaSet is constantly tested against

version transaction support
3.6.14 -
4.0.12 +
4.2.8 +
4.4.4 +
5.0.5 +

Examples

Click to see a single node example 
class ITTest {
    @Test
    void testDefaultSingleNode() {
        try (
            //create a single node mongoDbReplicaSet and auto-close it afterwards
            final MongoDbReplicaSet mongoDbReplicaSet = MongoDbReplicaSet.builder()
                .mongoDockerImageName("mongo:4.4.4")
                .build()
        ) {
            //start it
            mongoDbReplicaSet.start();
            assertThat(
                mongoDbReplicaSet.nodeStates(mongoDbReplicaSet.getMongoRsStatus().getMembers()),
                hasItem(ReplicaSetMemberState.PRIMARY)
            );
            assertNotNull(mongoDbReplicaSet.getReplicaSetUrl());
        }
    }
}
Click to see a fault tolerance example 
class ITTest {
    @Test
    void testFaultTolerance() {
        try (
            //create a PSA mongoDbReplicaSet and auto-close it afterwards
            final MongoDbReplicaSet mongoDbReplicaSet = MongoDbReplicaSet.builder()
                //with the latest mongo:4.4.4 docker image
                .mongoDockerImageName("mongo:4.4.4")
                //If true then use host.docker.internal of Docker, 
                //otherwise take dockerhost of Qoomon docker-host.
                //Make sure that your OS host file includes one of them.
                //All new Docker versions support the first variant.
                .useHostDockerInternal(true)
                //with 2 working nodes
                .replicaSetNumber(2)
                //with an arbiter node
                .addArbiter(true)
                //create a proxy for each node to simulate network partitioning
                .addToxiproxy(true)
                .build()
        ) {
            //start it
            mongoDbReplicaSet.start();
            assertNotNull(mongoDbReplicaSet.getReplicaSetUrl());

            //get a primary node
            final MongoNode masterNode = mongoDbReplicaSet.getMasterMongoNode(
                mongoDbReplicaSet.getMongoRsStatus().getMembers()
            );

            //cut off the primary node from network
            mongoDbReplicaSet.disconnectNodeFromNetwork(masterNode);
            //wait until a new primary is elected that is different from the masterNode
            mongoDbReplicaSet.waitForMasterReelection(masterNode);
            assertThat(
                mongoDbReplicaSet.nodeStates(mongoDbReplicaSet.getMongoRsStatus().getMembers()),
                hasItems(
                    ReplicaSetMemberState.PRIMARY,
                    ReplicaSetMemberState.ARBITER
                )
            );

            //bring back the disconnected masterNode
            mongoDbReplicaSet.connectNodeToNetwork(masterNode);
            //wait until all nodes are up and running
            mongoDbReplicaSet.waitForAllMongoNodesUp();
            assertThat(
                mongoDbReplicaSet.nodeStates(mongoDbReplicaSet.getMongoRsStatus().getMembers()),
                hasItems(
                    ReplicaSetMemberState.PRIMARY,
                    ReplicaSetMemberState.ARBITER,
                    ReplicaSetMemberState.SECONDARY
                )
            );
        }
    }
}

Motivation

  • Cross-platform solution that doesn't depend on fixed ports;
  • Testing MongoDB transactions to run against an environment close to a production one;
  • Testing production issues by recreating a real MongoDB replica set (currently without shards);
  • Education to newcomers to the MongoDB world (learning the behaviour of a distributed NoSQL database while dealing with network partitioning, analyze the election process and so on).

General info

Click to see how to create a 3 node replica set on fixed ports via Docker manually

MongoDB starting from version 4 supports multi-document transactions only on a replica set. For example, to initialize a 3 node replica set on fixed ports via Docker, one has to do the following:

  1. Add 127.0.0.1 mongo1 mongo2 mongo3 to the host file of an operation system;
  2. Run in terminal:
    • docker network create mongo-cluster
    • docker run --name mongo1 -d --net mongo-cluster -p 50001:50001 mongo:4.0.10 mongod --replSet docker-rs --port 50001
    • docker run --name mongo2 -d --net mongo-cluster -p 50002:50002 mongo:4.0.10 mongod --replSet docker-rs --port 50002
    • docker run --name mongo3 -d --net mongo-cluster -p 50003:50003 mongo:4.0.10 mongod --replSet docker-rs --port 50003
  3. Prepare the following unix end of lines script (optionally put it folder scripts or use rs.add on each node): 
    rs.initiate({
    "_id": "docker-rs",
    "members": [
        {"_id": 0, "host": "mongo1:50001"},
        {"_id": 1, "host": "mongo2:50002"},
        {"_id": 2, "host": "mongo3:50003"}
    ]
});
  4. Run in terminal:
    • docker cp scripts/ mongo1:/scripts/
    • docker exec -it mongo1 /bin/sh -c "mongo --port 50001 < /scripts/init.js"

As we can see, there is a lot of operations to execute and we even didn't touch a non-fixed port approach. That's where the MongoDbReplicaSet might come in handy.

Supported features

Feature Description default value how to set
replicaSetNumber The number of voting nodes in a replica set including a master one 1 MongoDbReplicaSet.builder()
awaitNodeInitAttempts The number of approximate seconds to wait for a master or an arbiter node(if addArbiter=true) 29 starting from 0 MongoDBReplicaSet.builder()
propertyFileName yml file located on the classpath none MongoDbReplicaSet.builder()
mongoDockerImageName a MongoDB docker file name mongo:4.0.10 finds first set:
1) MongoDbReplicaSet.builder()
2) the system property mongoReplicaSetProperties.mongoDockerImageName
3) propertyFile
4) default value
addArbiter whether or not to add an arbiter node to a cluster false MongoDbReplicaSet.builder()
slaveDelayTimeout whether or not to create one master and the others as delayed members false MongoDbReplicaSet.builder()
useHostDockerInternal If true then use host.docker.internal of Docker, otherwise take dockerhost of Qoomon docker-host false finds first set:
1) MongoDbReplicaSet.builder()
2) the system property mongoReplicaSetProperties.useHostDockerInternal
3) default value
addToxiproxy whether or not to create a proxy for each MongoDB node via Toxiproxy false MongoDbReplicaSet.builder()
enabled whether or not MongoReplicaSet is enabled even if instantiated in a test true finds first set:
1) the system property mongoReplicaSetProperties.enabled
2) propertyFile
3) default value
commandLineOptions command line options, example:Arrays.asList("--oplogSize", "50") emptyList MongoDbReplicaSet.builder()

a propertyFile.yml example:

mongoReplicaSetProperties:
  enabled: false
  mongoDockerImageName: mongo:4.1.13

License

Apache License, Version 2.0

Additional links

Copyright

Copyright (c) 2022 Konstantin Silaev [email protected]