Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Event System Overhaul & Cleanup #539

Open
wants to merge 13 commits into
base: develop
Choose a base branch
from
Open

Commits on Jan 29, 2023

  1. Event System Maturation:

    * Prepare for Job events
    * Prepare to replace hard-coded Workflow and Job events with more generic events, implemented the event system itself (instead of in the job and worklow systems)
    
    Events:
    * Simplified IEventData, removing Key generic parameter
    * Renamed eAuditType.eSceneQCd to eAuditType.ePubSceneQCd, to match naming conventions elsewhere
    * Initial, WIP versions of EventConsumerJob and EventConsumerWF
    * Added eJob and eWF eEventTopic's
    * Added eJob* and eWF* eEventKey's
    * Modified IEventEngine, replacing createConsumer() with registerConsumer() and unregisterConsumer()
    * Modified EventEngine, making it clear that we create a set of system consumers, one per event topic, and then register those via registerConsumer()
    jahjedtieson committed Jan 29, 2023
    Configuration menu
    Copy the full SHA
    41f329e View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2023

  1. Configuration menu
    Copy the full SHA
    a3fa4e1 View commit details
    Browse the repository at this point in the history
  2. Events:

    * Create a default event producer; expose it's send() method as a method of IEventEngine; this allows event publication without creation of an EventProducer.
    * Add eWFGenerateSceneDownloads event key
    * Remove IEventConsumer.poll, which was unused, and unnessarily complex
    
    Jobs:
    * Record Cook "step" during polling. When it changes from our previously recorded "Step", send an eJobUpdated event
    * Replace JobPackrat.updateEngines() with new sendEvent(), using events to inform the WorkflowEngine and JobEngine of job status changes
    * Replaced IJobEngine.jobCompleted by making the JobEngine an EventConsumer for Job events.  Use terminating statuses (done, error, cancelled) to clean up the job map
    
    Workflows:
    * Replace IWorkflowEngine.jobUpdated(), .event(), and .generateSceneDownloads() with actions initiated by job events.
    * Transform WorkflowEngine into a Job and Workflow Event consumer
    
    Cache & Data:
    * Removed vocabulary set and vocabulary for eWorkflowEvent
    
    Collections:
    * Use events to initiate scene download generation, instead of direct calls to the workflow engine
    
    GraphQL:
    * Use events to send the workflow ingest object notification
    
    Storage:
    * Use events to send the workflow ingest object notification from AssetStorageAdapter
    
    DBAPI:
    * Add "Step" field to JobRun, indicating the current step of a running job as received from Cook
    jahjedtieson committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    eaad91e View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2023

  1. Configuration menu
    Copy the full SHA
    05882f4 View commit details
    Browse the repository at this point in the history
  2. Build:

    * Used `npx madge --circular --extensions ts ./` to identfy circular dependencies in our code
    * Eliminated these through a combination of:
    - Using interfaces, where possible
    - Extracting selected code into separate files, which could then be imported directly without introducing the circ. dep.
    - Use forward declarations
    
    N.B. regressions tests are failing!
    jahjedtieson committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    72a74a6 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    1fa8c23 View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2023

  1. Storage:

    * Split part of OCFLObject into OCFLObjectBase, allowing OCFLInventory to access those parts of OCFLObject when OCFLObject needs OCFLInventory ... all without a circular dependency. Yay!
    jahjedtieson committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    b67ddf2 View commit details
    Browse the repository at this point in the history
  2. Event:

    * Avoid attempting to create a system consumer of Job and Workflow events -- there is no system consumer for these!
    jahjedtieson committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    94bfa1f View commit details
    Browse the repository at this point in the history
  3. DBAPI:

    * Extract from ObjectGraphDatabase those methods needed by ObjectGraph, in order to avoid a circular dependency between ObjectGraph and ObjectGraphDatabase
    jahjedtieson committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    83029ae View commit details
    Browse the repository at this point in the history
  4. Collections, Events, Jobs, HTTP:

    * Replace partial class definitions with base classes to avoid circular dependencies:
    * Move EdanCollection.computeLicenseInfo to PublishScene.computeLicenseInfo
    * Declare EventEngine.receive in EventEngineBase abstract class
    * Move Downloader.httpRoute to RouteBuilder.httpRoute
    * Import SitemapGenerator's real definition in download.ts
    * Declare JobEngine.sendJobEvent in JobEngineBase abstract class
    jahjedtieson committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    c0aaa74 View commit details
    Browse the repository at this point in the history
  5. Test:

    * Cache test does not need to use or test DBConnection.disconnect()!
    * Address new compilation issues (?)
    jahjedtieson committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    0d72465 View commit details
    Browse the repository at this point in the history
  6. Test:

    * For now, disable jest globalTeardown's call to prisma.$disconnect().
    jahjedtieson committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    2fc2183 View commit details
    Browse the repository at this point in the history
  7. HTTP:

    * Move Downloader.httpRoute to RouteBuilder.httpRoute
    jahjedtieson committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    3de3952 View commit details
    Browse the repository at this point in the history