Skip to content

Latest commit

 

History

History
369 lines (302 loc) · 13.3 KB

README.org

File metadata and controls

369 lines (302 loc) · 13.3 KB
Raw

Slouch

https://img.shields.io/clojars/v/com.balloneij/slouch.svg

An idiomatic Clojure interface to Apache CouchDB.

Table of Contents

Features

  • Ergonomic document API - Documents are treated like Clojure atoms.
    • swap!, deref, and reset! operate on CouchDB documents.
  • Minimizes bandwith

Installation

slouch is available from Clojars.

[com.balloneij/slouch "0.1.0"]

Quickstart

Everything you need is in the slouch.api namespace.

(ns app.core
  (:require [slouch.api :as slouch]))

Create a database. See API.

(with-open [db (slouch/database {:url "http://localhost:5984"
                                 :name "albums"
                                 :username "admin"
                                 :password "hunter1"})]
  ;; your code
  )

;; or

(slouch/with-database [db {:url "http://localhost:5984"
                           :name "albums"
                           :username "admin"
                           :password "hunter1"}]
  ;; your code
  )

Documents are like Clojure atoms. See API.

(let [doc (slouch/insert db {:sales 0})]
  (println @doc)
  ;; {:sales 0, :_id 5dde5901-6ee6-4084-a18e-b58f4487e4a2, :_rev 1-ea366df7bb92694d7de64184343c080e}

  (swap! doc update :sales inc)
  (println @doc)
  ;; {:sales 1, :_id 5dde5901-6ee6-4084-a18e-b58f4487e4a2, :_rev 2-68fb51089122a02a4d24f0910532b0f0}

  (reset! doc {:sales 0})
  (println @doc)
  ;; {:sales 0, :_id 5dde5901-6ee6-4084-a18e-b58f4487e4a2, :_rev 3-895e6de5e9418a64d7946247459bc769}

  (slouch/remove db (slouch/id doc))
  (println (slouch/exists? doc))
  ;; false
  )

Query views. See API.

(slouch/rows db :albums/by-artist {:key "ABBA"})
;; => [{:key "ABBA" :value {:name "Arrival"}}
;;     {:key "ABBA" :value {:name "Voyage"}}
;;     {:key "ABBA" :value {:name "Waterloo"}}]

(-> (slouch/row db :albums/by-name "Thriller" {:include-docs? true})
    (:doc)
    (swap! update :sales inc)
    (:sales))
;; => 70000001

API

Database

Slouch only interfaces with one database on a CouchDB server at a time.

;; Database is closeable. It's meant to be used in with-open i.e. try with resources
;; to ensure the underlying processes stop and thread pool resources get released.
(slouch/database {:url "http://localhost:5984" :name "albums"
                  :username "webapp-user" :password "super-secret"})

;; Shorthand for (with-open [db (slouch/database config)])
(with-database [db {:url "http://localhost:5984" :name "albums"
                    :username "webapp-user" :password "super-secret"}])

Database config

{;; CouchDB url.
 :url "http://localhost:5984"
 ;; The name of the database.
 :name "albums"
 ;; Credentials.
 :username "webapp-user"
 :password "super-secret"
 ;; Whether to allow insecure https connections. Default is false.
 :insecure? false
 ;; Size of thread pool for http connections to CouchDB. Default is 8.
 :pool-threads 8
 ;; Seconds to keep connections open before automatically closing them.
 ;; Default is 60 seconds.
 :pool-timeout 60
 ;; Milliseconds to wait before aborting a new connection attempt,
 ;; or 0, meaning no timeout (not recommended). Default is 5000 ms.
 :connection-timeout 5000
 ;; Milliseconds of data silence to wait before abandoning an established connection,
 ;; or 0, meaning no timeout (not recommended). Default is 5000 ms.
 :socket-timeout 5000
 ;; Seconds of session time remaining before reauthenticating.
 ;; Default is 60 seconds.
 :session-auth-threshold 60
 ;; Seconds remaining before considering a session expired. At a minimum,
 ;; consider setting this value greater than socket-timeout + connection-timeout.
 ;; Default 30 seconds.
 :session-timing-error 30
 ;; Milliseconds to keep a continuous connection open on /db/_changes to
 ;; watch for updates to documents stored in cache. A lower interval means
 ;; cache documents are added/removed to the watch more quickly to the watch list,
 ;; at the expense of reopening connections more frequently.
 ;; Default 10000 ms.
 :feed-refresh-interval 10000
 ;; Minutes to keep documents stored in memory.
 ;; Default 15 min.
 :cache-doc-ttl 15}

Documents

;; Insert new document with random uuid
(slouch/insert db {:name "21" :artist "Adele"})
;; Insert a new document with a specific id
(slouch/insert db "the-wall" {:name "The Wall" :artist "Pink Floyd"})

;; Get a document by id
(slouch/get db "abbey-road")
;; Get a document by id, or insert it if it does not exist
(slouch/get-or-insert db "spice" (fn [] {:name "Spice" :artist "Spice Girls"}))

;; Remove a document, no matter the revision
(slouch/remove db "the-wall")
;; Remove a document at specific revision
(slouch/remove db "the-wall" "3-2adcff8fb8b3f77825f627ad97464c80")

;; ID of a document
(slouch/id doc)
;; Revision of the current doc (or nil if it doesn't exist)
(slouch/rev doc)
;; Check if a document exists
(slouch/exists? doc)

;; Get a document from CouchDB
;; NOTE: Deref-ing will return the latest value unless called
;;       within a snapshot context. See "Ring middleware" for more details
(deref doc)
@doc

;; Like swapping a Clojure atom, but writes to CouchDB
(swap! doc assoc :genre ["pop" "post-disco" "funk" "rock"])

(let [old-val @doc
      new-val {:name "Thriller" :artist "Michael Jackson"}]
  ;; Set a new value iff the :_rev from an old value matches the rev
  ;; of the current document in CouchDB
  (compare-and-set! doc old-val new-val))

;; Like reseting a Clojure atom, but writes to CouchDB
(reset! doc {:name "Thriller" :artist "Michael Jackson"})

Views

(slouch/view db ddoc-view opts) is the main interface for querying CouchDB views. Additional functions are provided to make working with the results more ergonomic.

To query a view, provide the design document and view name by one of two means:

  • a vector ["design-doc" "view-name"].
  • a namespaced keyword :design-doc/view-name.

All view functions take view options.

;; Query a view for :offset, :rows, and :total-rows. See "View options"
(slouch/view db :albums/by-name)
(slouch/view db :albums/by-name {:skip 20})

;; Equivalent to (:rows (slouch/view db ddoc-view opts))
(slouch/rows db :albums/by-certification)
(slouch/rows db :albums/by-certification {:key "platinum"})

;; Equivalent to (first (:rows (slouch/view db ddoc-view (merge opts {:key k :limit 1}))))
(slouch/row db ["albums" "by-name"] "Millennium")
(slouch/row db ["albums" "by-name"] "Millennium" {:include-docs? true})

;; Equivalent to (->> (slouch/view db ddoc-view opts)
;;                    :rows
;;                    (map :doc))
(slouch/docs db :albums/by-name)
(slouch/docs db :albums/by-name {:start-key "1" :end-key "Thriller"})

;; Equivlanet to
;; (-> (view db ddoc-view (merge opts {:key k
;;                                     :limit 1
;;                                     :include-docs? true}))
;;     :rows
;;     first
;;     :doc)
(slouch/doc db :albums/by-name "Thriller")
(slouch/doc db :albums/by-name "Thriller" {:stable? true})

View options

View options come directly from the CouchDB view endpoint.

{;; Include conflicts information in response. Ignored if include-docs isn’t true. Default is false.
 :conflicts? false
 ;; Return the documents in descending order by key. Default is false.
 :descending? false
 ;; Stop returning records when the specified key is reached.
 :end-key {:name "wish-you-were-here"}
 ;; Stop returning records when the specified document ID is reached. Ignored if end-key is not set.
 :end-key-doc-id "255ce80b1928875f253f5fca670d0599"
 ;; Group the results using the reduce function to a group or single row. Implies reduce is true and the maximum group-level. Default is false.
 :group? false
 ;; Specify the group level to be used. Implies group is true.
 :group-level 2
 ;; Include the associated document with each row. Default is false.
 :include-docs? false
 ;; Specifies whether the specified end key should be included in the result. Default is true.
 :inclusive-end? true
 ;; Return only documents that match the specified key.
 :key {:name "boston"}
 ;; Return only documents where the key matches one of the keys specified in the array.
 :keys [{:name "millennium"} {:name "like-a-virgin"} {:name "purple-rain"}]
 ;; Limit the number of the returned documents to the specified number.
 :limit 20
 ;; Use the reduction function. Default is true when a reduce function is defined.
 :reduce? true
 ;; Skip this number of records before starting to return the results. Default is 0.
 :skip 0
 ;; Sort returned rows. Setting this to false offers a performance boost. The total-rows and offset fields are not available when this is set to false. Default is true.
 ;; See Sorting Returned Rows https://docs.couchdb.org/en/stable/api/ddoc/views.html#sorting-returned-rows
 :sorted? true
 ;; Whether or not the view results should be returned from a stable set of shards. Default is false.
 :stable? false
 ;; Return records starting with the specified key.
 :start-key {:name "baby-one-more-time"}
 ;; Return records starting with the specified document ID. Ignored if startkey is not set.
 :start-key-doc-id "255ce80b1928875f253f5fca670d3e15"
 ;; Whether or not the view in question should be updated prior to responding to the user. Supported values: true, false, :lazy. Default is true.
 :update true
 ;; Whether to include in the response an update-seq value indicating the sequence id of the database the view reflects. Default is false.
 :update-seq? false}

Unsupported Options

{;; Include the Base64-encoded content of attachments in the documents that are included if include-docs is true. Ignored if include-docs isn’t true. Default is false.
 :attachments? false
 ;; Include encoding information in attachment stubs if include-docs is true and the particular attachment is compressed. Ignored if include-docs isn’t true. Default is false.
 :att-encoding-info? false
 ;; Deprecated by CouchDB. Use :stable and :update instead.
 ;;  :ok is equivalent to {:stable true :update false}
 ;;  :update_after is equivalent to {:stable true :update lazy}
 ;; The default behavior is equivalent to {:stable false :update true}.
 :stale :ok}

Ring middleware

wrap-db handles each request inside a snapshot.

Inside a snapshot, the value of a document will stay the same throughout the duration of a request, unless an update occurs within the same snapshot.

Therefore, don’t be afraid to deref a document multiple times within a single request. At most, the document will be fetched from CouchDB one time.

(slouch/with-database [db config]
  (-> handler
      ;; Add :db to incoming requests and execute handler inside a snapshot context
      (slouch/wrap-db db)
      ;; or use a different key
      (slouch/wrap-db :my-db db)
      (run-webapp)))

Limitations

  • No means for solving document conflicts.
  • Cannot handle document attachments.
  • No means for seamless failover to other CouchDB instances.
  • Cannot solve world hunger.

Goals

  • Encode username and password so they aren’t stored in mem as plaintext

    In case somewhere, somehow the db config gets prn-str‘ed (logs, stacktraces, etc.), it would be best if the username and password were at least base64 encoded.

    Maybe hide the values inside record and define a print-method to hide the password.

  • Add a size limit to documents added to cache
  • Reducible, transducer-ready view result

    next-jdbc provides next.jdbc/plan which is a cool way to stream and process incoming SQL results. It could be fun to expirement with a similar system for Slouch and test to see if it has any merit speeding up view queries.

  • Lazily get ~rows~ It could be more efficiently to paginate rows results. For example, limit 100 records and then lazy-seq to get more.
  • Multiple CouchDB instances

    Support multiple CouchDB instances doing master-slave replication.

    i.e.

    • 1 master - write-only
    • N replicas - read-only

    A DBA could locate replicas at the same datacenters/device as the client, and then host the master in a central location.

  • Support document attachments

Changelog

  • 0.1.0 Initial release

License

Copyright 2023 Isaac Ballone.

Distributed under the MIT License.