by David SchmüddeJul 30 2019
Turing complete.

Say What You Mean

Me

  • 🎓 Computer Science → Music
  • 🎥 Time-Based Media
  • 💻 Nextjournal

@dschmudde . http://schmud.de/

Artists + Programmers

Artists and programmers are in the business of representation and abstraction.

Contemporary Art

Object

The human scale.

What does it mean or what does it do?

Environment

The scale of reality.

Can it be manipulated?

Art

← 19th Century Art

  • Rendering accuracy
  • Dramatic depictions
  • Depiction of the commonplace
  • Quality of the object's attributes (color, texture, ...)

→ 20th Century Art

  • The medium of painting itself
  • The artist's metaphysical state

Contemporary Art

The art object intrinsically feels more knowable and controlled than the conditions that determine how art is experienced. The environment.

Models

The Computer as a Communication Device

J.C.R. Licklider and Robert W. Taylor (1968)

Computer Modeling → Communication

  • Insight into the behavior of a system → insight into another person's mind.
  • Includes presumptions, unconscious biases, factual failings, and intuitive intelligence.

Hubris in Modeling

As mid-century artists acknowledge the greater forces of the environment, computer scientists endeavor to model the unknown and promise big returns.

  • Natural language processing of the 1960s
  • The Computer as a Communication Device

Object Oriented Programming

The model is the code. The object is the state.

class Post:

    __max_size = 280
    text = ""

    def __init__(self, text):
        self.text = text

    def __validate_size(self):
        if(len(self.text) <= self.__max_size):
            return True
        else:
            return False

    def send(self):
        if self.__validate_size():
            return "API CALL: " + self.text
        else:
            return "API CALL: Error - text too long"
tweet = Post("lorem ipsum")
tweet.send()
'API CALL: lorem ipsum'
tweet = Post(" lorem ipsum lorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsumlorem ipsum")
tweet.send()
'API CALL: Error - text too long'

Specifications

The model validates the code. The data is the state.

Function

(ns user)

(defn api-call [text video]
  (str "API CALL: " text " + " video))

(defn send-post [text & opts] 
   (let [{:keys [video sound image]} opts]
     (user/api-call text video)))
user/send-post

Specification

(ns user)

;; Specification
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as stest])

(s/fdef send-post
    :args (s/cat :text ::valid-text
                 :variadic (s/* any?)))
(stest/instrument `send-post)

(str "Setup Specification #1 for " *ns*)
"Setup Specification #1 for user"
(s/def ::valid-text (s/and string? #(<= (count %) 280)))
:user/valid-text

Call

(send-post "lorem ipsum" :video "xyz.mov")
"API CALL: lorem ipsum + xyz.mov"
(try 
  (send-post "Voluptatem aspernatur sit aliquid. Officiis quo ut impedit hic explicabo et laudantium. Ut eos expedita est aut iure at.Voluptatem corporis et laborum totam incidunt earum et. Aut ut nemo nisi esse aliquam est error. Et inventore sunt itaque dignissimos. Voluptas natus quam reprehenderit iure consectetur nulla. Laudantium molestias eos delectus. Odit quia nesciunt consequatur quia et omnis dolores. Voluptas accusamus occaecati ex quo porro. Dolor consequatur alias m." :video "xyz.mov")
  (catch Exception e (str "ERROR: " (.getMessage e))))
"ERROR: Call to #'user/send-post did not conform to spec."

Environment #2

(ns wordpress)

;; Specification
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as stest])

(s/fdef send-post
    :args (s/cat :text ::valid-text
                 :variadic (s/* any?)))
(stest/instrument `send-post)

(str "Setup Specification #2 for " *ns*)
"Setup Specification #2 for wordpress"
(ns wordpress)

(defn send-post [text & opts] 
   (let [{:keys [video sound image]} opts]
     (user/api-call text video)))

(s/def ::valid-text (s/and string? #(> (count %) 0)))
:wordpress/valid-text

Call

(ns wordpress)

(try 
  (send-post "Voluptatem aspernatur sit aliquid. Officiis quo ut impedit hic explicabo et laudantium. Ut eos expedita est aut iure at.Voluptatem corporis et laborum totam incidunt earum et. Aut ut nemo nisi esse aliquam est error. Et inventore sunt itaque dignissimos. Voluptas natus quam reprehenderit iure consectetur nulla. Laudantium molestias eos delectus. Odit quia nesciunt consequatur quia et omnis dolores. Voluptas accusamus occaecati ex quo porro. Dolor consequatur alias m." :video "xyz.mov")
  (catch Exception e (str "Catch exception" (.getMessage e))))
"API CALL: Voluptatem aspernatur sit aliquid. Officiis quo ut impedit hic explicabo et laudantium. Ut eos expedita est aut iure at.Voluptatem corporis et laborum totam incidunt earum et. Aut ut nemo nisi esse aliquam est error. Et inventore sunt itaque dignissimos. Voluptas natus quam reprehenderit iure consectetur nulla. Laudantium molestias eos delectus. Odit quia nesciunt consequatur quia et omnis dolores. Voluptas accusamus occaecati ex quo porro. Dolor consequatur alias m. + xyz.mov"

The the function's behavior reveals something about its environment.

Declarative Statements

I want to express my feelings rather than illustrate them.

~ Jackson Pollock

Process

  • Artists follow intuition
  • Intuition is not always logical or sequential
  • Results can be analyzed

Analysis

Process

  • Declarative statements are better for exploring
  • Imperative approaches are taken arbitrarily early on

The Execution Environment

Computer as a Communication Device

Objective-C/NeXTSTEP

  • Where is the object state?
  • Where are the messages?

Objective-C → Swift

SwiftUI uses a declarative syntax so you can simply state what your user interface should do. For example, you can write that you want a list of items consisting of text fields, then describe alignment, font, and color for each field. Your code is simpler and easier to read than ever before, saving you time and maintenance.

import SwiftUI

struct Content : View {
  
  @State var model = Themes.listModel
  
  var body: some View {
  	List(model.items, action: model.selectItem) { item in 
      Image(item.image)
      VStack(alignment: .leading) {
          Text(item.title)
  				Text(item.subtitle)
  					.color(.gray)
  }}}}
Swift

Describing the Environment

O̶b̶j̶e̶c̶t̶s̶ Data

(dissoc @re-frame.db/app-db :auth-token)
(count (-> (get-in @re-frame.db/app-db [:article :nodes]) vals))

M̶e̶t̶h̶o̶d̶s̶ Pure Functions

(let [id "578ef919-75a4-473f-bf52-5c2fdde45af4"]
	(re-frame.core/dispatch [:insert-code-node id]))

Code

How does :insert-code-node work? Nextjournal in Nextjournal details how to use grep to scan 80,000+ lines of code, grab the function, and run it.

The :insert-code-node dispatch uses seven functions from the namespace com.nextjournal.editor.operations.tree, included here:

Shift+Enter to run
ClojureScript
(ns tree)

(defn root-id [db]
  (get-in db [:article :root]))

(defn root? [db node-id]
  (= node-id (root-id db)))

(defn parent-by-id [db node-id content-key]
  (->> db
       :article
       :nodes
       vals
       (filter #(some #{node-id} (content-key %)))
first))

(defn full-position-by-id
  "Returns a vector describing the position of the node in the form
  [parent-id content-key position]
  E.g
  [\"abc-123\" :nodes 2]
  Returns nil if node-id is the root node"
  [db node-id]
  (when-not (root? db node-id)
    (let [[parent content-key] (some (fn [key]
                                       (when-let [p (parent-by-id db node-id key)]
                                         [p key]))
                                     [:content :sections :nodes :inlines])
          content-pos (.indexOf (vec (content-key parent)) node-id)]
      (assert parent)
      (assert (<= 0 content-pos))
[(:id parent) content-key content-pos])))

(defn split-at-item [v i]
  (split-with #(not= % i) v))

(defn flat
  ([db] (assert db) (flat (get-in db [:article :nodes]) (get-in db [:article :root])))
  ([all-nodes current-node-id]
   (assert all-nodes)
   (let [node (all-nodes current-node-id)]
     (concat [current-node-id]
             (:content node)
(mapcat (partial flat all-nodes) (:sections node))))))

(defn closest-code-cell
  "Given an `article` and an absolute position `pos`, tries to find a code cell
   above the given position first. If there's no matching cell found above,
   it returns the closes below, nil otherwise. Accepts an optional predicate
   which all code cells considered should satisfy."
  ([article pos]
   (closest-code-cell article pos (constantly true)))
  ([article [section-id content-key idx :as pos] pred]
   (let [db {:article article}
         contents (vec (get-in article [:nodes section-id content-key]))
         node-at-pos (if (contains? contents idx)
                       (get contents idx)
                       (get-in article [:nodes section-id :sections 0]))
         flat (flat db)
         [before after] (split-at-item flat node-at-pos)
         closest-nodes (concat (reverse before) after)]
     (some
      (fn [id] (let [node (get-in article [:nodes id])]
                 (when (and (pred node) (= "code" (:kind node)))
                   node)))
closest-nodes))))

"[com.nextjournal.editor.operations.tree :as tree]"

Exploration

The :insert-code-node dispatch is just an anonymous function and can be run simply by passing in the re-frame database and a valid node-id for modification. It will return the dispatch, which is just data.

((fn [{:keys [db] :as ctx} [_ node-id attrs]]
   (let [db (dissoc db :auth-token)
         [p o s :as pos] (full-position-by-id db node-id)
         find-closest-lang (comp :language
                                 (partial tree/closest-code-cell (:article db)))
         attrs (update attrs :language #(or % (find-closest-lang pos) "julia"))]
     {:db db :dispatch {:insert-code-node-into p o s "code" attrs}}))
  {:db @re-frame.db/app-db} ["" "578ef919-75a4-473f-bf52-5c2fdde45af4"])

Call

Now that the dispatch better understood, call send-post within the context of a new environment. In this case, a Nextjournal article. Running the cell will modify the article itself.

(defn api-call [text id kind]
  (re-frame.core/dispatch [:insert-after id kind {:content text}]))

(defn send-post [text & opts] 
   (let [{:keys [video sound image id kind]} opts]
     (api-call text id kind)))

(send-post "lorem ipsum" :id "d404003e-ea11-41df-87c5-d21ef9392ba1" :kind "paragraph")

lorem ipsum

Referential Transparency

Actual space is intrinsically more powerful and specific than paint on a flat surface.

~ Donald Judd

The painted depreciation is an object that points to the environment. It is neither specific enough to encapsulate the environment nor is it detailed enough to embody the environment.

  • Data makes no attempt to be complete.
  • Selecting data is an act of curation.
  • When and how a simple function is invoked reveals complex choices.

Thank You

Say What You Mean: Exploring Declarative Computation in Art

@dschmudde . http://schmud.de

https://nextjournal.com/schmudde/