Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add transaction recoverability #43572

Closed
wants to merge 76 commits into from
Closed

Add transaction recoverability #43572

wants to merge 76 commits into from

Commits on Jan 31, 2024

  1. Implement recovery log file system

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    3106e32 View commit details
    Browse the repository at this point in the history

  2. Cleanup unused imports

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    630b4c5 View commit details
    Browse the repository at this point in the history

  3. Add initialization of fileRecoveryLog to TransactionResourceManager

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    9a56cdf View commit details
    Browse the repository at this point in the history

  4. Add TransactionRecord to store a log record

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    1cb2ff8 View commit details
    Browse the repository at this point in the history

  5. Implement in memory log and rewrite some functions

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    d295306 View commit details
    Browse the repository at this point in the history

  6. Add enum for recovery statuses instead of string

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    a540ae5 View commit details
    Browse the repository at this point in the history

  7. Add recoveryLogName config to transaction.bal

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    f8a2b3f View commit details
    Browse the repository at this point in the history

  8. Fix null pointer and concurrent modification exceptions with existing… 

    …Logs
    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    4b6dfdb View commit details
    Browse the repository at this point in the history

  9. Updated comments

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    bb0df43 View commit details
    Browse the repository at this point in the history

  10. Add hazard state to RecoveryStatus

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    228fb0b View commit details
    Browse the repository at this point in the history

  11. Implement checkpointing for file and in memory logs

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    8f8418d View commit details
    Browse the repository at this point in the history

  12. Change default checkpoint interval

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    c11d3e4 View commit details
    Browse the repository at this point in the history

  13. Move parseTransactionLogRecord to TransactionLogRecord class

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    a51ee30 View commit details
    Browse the repository at this point in the history

  14. Add configs for log directory and deletion of old logs

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    38b0442 View commit details
    Browse the repository at this point in the history

  15. Add comments and cleanup code

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    36f1c79 View commit details
    Browse the repository at this point in the history

  16. Add force/lazy writing to logs

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    6f78138 View commit details
    Browse the repository at this point in the history

  17. Update recovery statuses

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    c49e433 View commit details
    Browse the repository at this point in the history

  18. Update Xid to gtrid and unique formatId

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    774c527 View commit details
    Browse the repository at this point in the history

  19. Add a placeholder bqual

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    af0dbe4 View commit details
    Browse the repository at this point in the history

  20. Implement recovery for xaResources

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    86a1e35 View commit details
    Browse the repository at this point in the history

  21. Refactor RecoveryStatus to RecoveryState

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    0d2fc9f View commit details
    Browse the repository at this point in the history

  22. Update TransactionLogRecord to include log time

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    dab23b5 View commit details
    Browse the repository at this point in the history

  23. Update createXID to use trxId and blockId

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    86ae597 View commit details
    Browse the repository at this point in the history

  24. Update hazard state log record

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    30478f8 View commit details
    Browse the repository at this point in the history

  25. Handle exceptions in xaResourceRecovery

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    d8bcba2 View commit details
    Browse the repository at this point in the history

  26. Minor changes to logs

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    152d2ab View commit details
    Browse the repository at this point in the history

  27. Implement the log manager

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    83fe1e2 View commit details
    Browse the repository at this point in the history

  28. Add getDefaultFormat in XIDGenerator

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    24b1c3f View commit details
    Browse the repository at this point in the history

  29. Write terminated logs during transaction clean up

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    81a8c92 View commit details
    Browse the repository at this point in the history

  30. Update getCombinedId

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    23b10e6 View commit details
    Browse the repository at this point in the history

  31. Remove placeholder branch qualifier const

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    330cbb2 View commit details
    Browse the repository at this point in the history

  32. Refactor method name in LogManager

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    c797299 View commit details
    Browse the repository at this point in the history

  33. Changes to the recovery logic 

    This commit changes the initial logic where you check the log and take actions according to that on xa resource to other way around by getting the failed transactions in xa resources and taking the action from the log. This way we can ensure that there are no unnecessary commit/rollback calls made to the xa resources which would return invalid xid errors.
    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    f9269fa View commit details
    Browse the repository at this point in the history

  34. Add startupCrashRecover to transaction-block in desugar

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    db98c84 View commit details
    Browse the repository at this point in the history

  35. Remove unused method

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    bf7d6e2 View commit details
    Browse the repository at this point in the history

  36. Handle transaction recovery errors and warnings

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    ea89fbb View commit details
    Browse the repository at this point in the history

  37. Handle prepared transactions without a decision log record

    @dsplayerX
    dsplayerX committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    0524380 View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2024

  1. Add licence headers

    @dsplayerX
    dsplayerX committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    8bcafe5 View commit details
    Browse the repository at this point in the history

  2. Update switch cases in RecoveryManager

    @dsplayerX
    dsplayerX committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    4fd4179 View commit details
    Browse the repository at this point in the history

  3. Fix formatting

    @dsplayerX
    dsplayerX committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    5c63a68 View commit details
    Browse the repository at this point in the history

  4. Remove unnecessary conditions and variables

    @dsplayerX
    dsplayerX committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    d629dbd View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2024

  1. Move inline values to dedicated constants.

    @dsplayerX
    dsplayerX committed Feb 2, 2024
    Configuration menu
    Copy the full SHA
    6d2287d View commit details
    Browse the repository at this point in the history

  2. Remove redundant checks in FileRecoveryLog

    @dsplayerX
    dsplayerX committed Feb 2, 2024
    Configuration menu
    Copy the full SHA
    a33fb3c View commit details
    Browse the repository at this point in the history

  3. Improve errors in FileRecoveryLog

    @dsplayerX
    dsplayerX committed Feb 2, 2024
    Configuration menu
    Copy the full SHA
    9758540 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2024

  1. Address review suggestions

    @dsplayerX
    dsplayerX committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    b814905 View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2024

  1. Replace switch cases with enhanced switch cases

    @dsplayerX
    dsplayerX committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    144b3d0 View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2024

  1. Fix import formatting

    @dsplayerX
    dsplayerX committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    87a031b View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2024

  1. Merge remote-tracking branch 'upstream/master' into feature-transacti… 

    …on-recovery
    @dsplayerX
    dsplayerX committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    c2af735 View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2024

  1. Add doc comments

    @dsplayerX
    dsplayerX committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    3019503 View commit details
    Browse the repository at this point in the history

  2. Rework recovery logic 

    The redundant "STARTING" log records were removed and the recovery log was reworked according to the log centric recovery approach.
    @dsplayerX
    dsplayerX committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    07084e3 View commit details
    Browse the repository at this point in the history

Commits on Mar 11, 2024

  1. Cleanup code 

    - Removed unnecessary variable creations
- Refactor some methods
    @dsplayerX
    dsplayerX committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    0d798fd View commit details
    Browse the repository at this point in the history

  2. Handle invalid filenames in log directory

    @dsplayerX
    dsplayerX committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    73be34d View commit details
    Browse the repository at this point in the history

  3. Remove trailing whitespaces when parsing logs

    @dsplayerX
    dsplayerX committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    c07cdaa View commit details
    Browse the repository at this point in the history

  4. Add cannot parse log record runtime error

    @dsplayerX
    dsplayerX committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    eb0775a View commit details
    Browse the repository at this point in the history

Commits on Mar 12, 2024

  1. Make startup recovery symbol a constant

    @dsplayerX
    dsplayerX committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    9807001 View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2024

  1. Add method to write multiple logs to file

    @dsplayerX
    dsplayerX committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    f303138 View commit details
    Browse the repository at this point in the history

  2. Remove redundant file name check in findLatestVersion

    @dsplayerX
    dsplayerX committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    6c02229 View commit details
    Browse the repository at this point in the history

  3. Add unit tests for recovery and log managers. 

    - Introduced mockito as a dependency.
- Added unit tests for RecoveryManager and LogManager.
    @dsplayerX
    dsplayerX committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    5ae4915 View commit details
    Browse the repository at this point in the history

  4. Add unit test for xid creation

    @dsplayerX
    dsplayerX committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    ad065ed View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2024

  1. Rename checkpoint interval key variable

    @dsplayerX
    dsplayerX committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    4b73d66 View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. Add method comments and fix typos

    @dsplayerX
    dsplayerX committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    ab61d60 View commit details
    Browse the repository at this point in the history

  2. Add unit tests for heuristic termination

    @dsplayerX
    dsplayerX committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    0d38a71 View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2024

  1. Make initAppendChannel thread safe 

    Previously, concurrent calls to createNextVersion() or put() could result in race conditions where multiple threads could attempt to initialize the appendChannel simultaneously. This was addressed by adding a double-checked locking mechanism to ensure that only one thread initializes the appendChannel when it's null.
    @dsplayerX
    dsplayerX committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    be64fab View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. Refactor createXid method

    @dsplayerX
    dsplayerX committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    878cc73 View commit details
    Browse the repository at this point in the history

  2. Resolve merge conflicts

    @dsplayerX
    dsplayerX committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    db1d550 View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. Make getFailedTransaction method concurrent safe

    @dsplayerX
    dsplayerX committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    d6a6919 View commit details
    Browse the repository at this point in the history

  2. Refactor handleHeuristicTermination method

    @dsplayerX
    dsplayerX committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    8232e93 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2024

  1. Refactor handling recovery in resources

    @dsplayerX
    dsplayerX committed May 9, 2024
    Configuration menu
    Copy the full SHA
    927c6b5 View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. Introduce FileLockAndChannel to handle append channel and locking

    @dsplayerX
    dsplayerX committed May 13, 2024
    Configuration menu
    Copy the full SHA
    11ac5e8 View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. Rework putAll method

    @dsplayerX
    dsplayerX committed May 27, 2024
    Configuration menu
    Copy the full SHA
    45e30eb View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2024

  1. Refactor transaction log managers to use singleton pattern

    @dsplayerX
    dsplayerX committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    393d187 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. Resolve merge conflicts

    @gimantha
    gimantha committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    5c63125 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/ballerina-platform/baller… 

    …ina-lang into dsplayerX-feature-transaction-recovery
    @gimantha
    gimantha committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    d202e3b View commit details
    Browse the repository at this point in the history

  3. Merge pull request #2 from gimantha/dsplayerX-feature-transaction-rec… 

    …overy

Sync with master
    @dsplayerX
    dsplayerX authored Oct 9, 2024
    Configuration menu
    Copy the full SHA
    224eea5 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. Merge pull request #42080 from dsplayerX/feature-transaction-recovery 

    Implement Transaction Recovery for XA Resources
    @gimantha
    gimantha authored Oct 11, 2024
    Configuration menu
    Copy the full SHA
    86ae40b View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2024

  1. Resolve merge conflicts

    @gimantha
    gimantha committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    c9ee85f View commit details
    Browse the repository at this point in the history