Spec integration

[yogthos]

Now that Spec is being added in Clojure 1.9, would it make sense for Schema to target it?

It seems like Spec is a superset of Schema functionality, but I think that Schema provides a much cleaner syntax for describing data. If Schema syntax would be used to generate Spec, then a lot of the complexity from the library could be offloaded to the standard Clojure library at that point.

[w01fe]

FWIW I don't think Spec is a complete superset of Schema yet (e.g. completion) but it is close.

The model for schema and spec is different enough (key registry, etc) that I think it could be nontrivial to use spec as a backend. And moreover (since part of the point of spec is to rally the community around a single way of writing schemas/specs), I'm not sure its a good idea to encourage people to write spec with an alternate syntax, especially one that doesn't necessary line up with its semantics (but thanks for your appreciation of Schema's syntax :)). We also want to keep compatibility with earlier Clojure versions for some amount of time, so couldn't replace the backend now regardless of the other points.

If there was to be some connection between Schema and Spec, my initial thought (without having used spec yet myself) is that the most useful might be a 'Schema to Spec converter'; and a 'Schema with Spec Backend' as you suggest might be useful as a temporary shim (e.g. replacement library for schema) while refactoring from Schema to Spec or using third-party Schematized libraries with Spec. But, I think both of those would probably be separate projects from schema (and I don't think I personally have the bandwidth to work on either of those right now), although we'd of course be happy to link to them from the README if someone else took a crack.

[yogthos]

Thanks for the thoughtful response. The idea came from this discussion, and Alex Miller is actually supportive of a Schema to Spec converter. I definitely agree that it would be a non-trivial project however. In my view, Spec doesn't really make Schema obsolete as it's focus is more general. Schema provides a very intuitive way to describe data, and I think there's a lot of value in its syntax. Hopefully both will coexist in the future. :)

[ikitommi]

Related: plumatic/plumbing#126. Will comment on that when out of the hammock ;)

[mattiasw2]

The map description in spec is more reusable than the one in schema.

However, spec's s/cat etc is nice, but actually overkill if you have simple functions with fixed set of arguments.

So, being able to write something like this would be really nice

(clojure.spec/def ::foo int?)
(schema.core/defn bar :- ::foo [x :- ::foo] nil)

where I just say that the function takes a ::foo and returns a ::foo

[ikitommi]

My 2 cents:

• add a new metadata for both schema & plumbing fns (and fnks), :fn-validation with :schema as the only supported value

• the schematised fn implementation calls a new multimethod (schema.core/fn-validation etc.) with the :fn-validation value and get back a Type satisfying a new FnValidation protocol: used to do the actual validation (if turned on) and fn-schema extraction.

• :default maps to :schema, which returns a SchemaFNValidation Type.

• to use Plumbing/Schema-fns with spec, one would need to add a dispatch function for :spec, which would return a SpecFNValidation type doing the validation & model extraction for spec

• for 100% spec apps, one could override the schema.core/fn-validation :default to point to :spec

We could host the SpecFNValidation on the spec-tools side to keep Schema & Plumbing spec-free.

;; implicit schema fn-validation
(schema.core/defn foo :- Long [a :- Long] a)

;; explicit schema fn-validition
(schema.core/defn ^{:fn-validation :schema} foo :- Long [a :- Long] a)

;; adds :spec as valid fn-validation (mutable evil)
(require '[spec-tools.schema])

(clojure.spec.alpha/def ::id int?)

;; explicit spec fn-validation
(schema.core/defn ^{:fn-validation :spec} foo :- int? [a :- ::id] a)

;; override the default (mutable evil 2)
(defmethod schema.core/fn-validation :default [_] spec-tools.schema/SpecFNValidation)

;; implicit spec fn-validation
(schema.core/defn foo :- int? [a :- ::id] a)

What do you think?

btw, defining fns (and fnks) with spec works, they just don't implement Schema protocols, so they fail at runtime:

(require '[clojure.spec.alpha :as s])

(s/def ::foo int?)

(schema.core/fn-schema
  (schema.core/fn [a :- ::foo]))
; IllegalArgumentException No implementation of method: :explain of protocol: #'schema.core/Schema found for class: clojure.lang.Keyword  clojure.core/-cache-protocol-fn (core_deftype.clj:583)

(schema.core/fn-schema
  (schema.core/fn [a :- int?]))
; IllegalArgumentException No implementation of method: :explain of protocol: #'schema.core/Schema found for class: clojure.core$int_QMARK_  clojure.core/-cache-protocol-fn (core_deftype.clj:583)

(schema.core/fn-schema
  (schema.core/fn [a :- (s/spec int?)]))
; IllegalArgumentException No implementation of method: :explain of protocol: #'schema.core/Schema found for class: clojure.spec.alpha$spec_impl$reify__751  clojure.core/-cache-protocol-fn (core_deftype.clj:583)

I have a custom reader for fn-schema -> spec now in compojure-api.

;; schema
(POST "/plus" []
  :query-params [b :- s/Int, {c :- s/Int 0}]
  :body [numbers {:d s/Int}]
  :return {:total s/Int}
  (ok {:total (+ a b c (:d numbers))}))

;; (data-)specs
(POST "/plus" []
  :query-params [b :- spec/int?, {c :- spec/int? 0}]
  :body [numbers {:d spec/int?}]
  :return {:total ::total}
  (ok {:total (+ a b c (:d numbers))})

[w01fe]

Thanks for the detailed proposal! To be up-front, I don't have the opportunity to do much Clojure these days, so any effort to add significant new functionality probably needs to be community-driven. As such, it might make sense to take this to the mailing list to see if others have thoughts.

With that out of the way, can you say a little more about the goals of this effort? Do you just want a way to use specs with Schema s/defn syntax, or is there more to it than that?

[ikitommi]

Thanks for the quick response.

I would like to have same kind of boilerplate-free spec definitions for functions like Schema & Plumbing do for Schema. Many people have been talking about this in Slack etc.

I guess there are 2 options:

1. make an extension point to Schema to to support other modelling libs

• would be a small effort (I could try to do this)
• tools like Cursive does already static analysis for both Schema & Plumbing

2. copy the great work you have done to another "spec-"project (e.g. spec-tools)

• redone spec-fnk syntax could have better support for namespaced keywords (Support for namespaced keys, part II plumbing#126)
• would still need extension points as I wan't to support both vanilla specs & data-specs (and others)
• big effort (I at least probably don't have time for this)

[w01fe]

Yeah, I guess my gut feeling is that if what you want is really another way to declare spec'd functions (without any use of the schema backend), that probably belongs in another library. I guess ideally that could be accomplished by splitting the schema syntax stuff into a separate library from the schema backend, but it sounds like neither of us have the bandwidth to tackle that.

So back to reality, I guess I'm open to considering something like (1) so long as it doesn't interfere with the current applications or ergonomics of schema, someone else commits to test and maintain it, and it won't lead to a lot of rough edges for people that try to use it. (How will it play with the existing schema tools for coercion/generation, if at all, etc?).

Another related question is whether you are hoping that under (1) spec-schema declarations will also work with plumbing and fnk out of the box. I think that unfortunately that will not be the case, since plumbing actually encodes knowledge about the structure of schemas (see, e.g., https://github.com/plumatic/plumbing/blob/master/src/plumbing/fnk/schema.cljx). Do you see an easy way around that?

Honestly I think just copying schema and deleting everything that's not the syntax, then modifying from there seems like it might be the easiest approach. But if you really think (1) is the way to go I'm happy to keep talking things through.

[ikitommi]

There is a Plumatic Schema (and plain clojure) syntax parser in malli nowadays: it doesn't care about the actual schema, just collects those. It emits malli schemas, but could be made configurable via a new Emitter protocol etc.

If someone is interested in making a schema-spec-malli converter or one defn for all, that's one extra tool to do think about.

code: https://github.com/metosin/malli/blob/master/src/malli/destructure.cljc

tests: https://github.com/metosin/malli/blob/master/test/malli/destructure_test.cljc