source: project/wiki/eggref/4/glls @ 30947

Last change on this file since 30947 was 30947, checked in by acharlton, 6 years ago

Grammar, clarification

File size: 23.4 KB
Line 
1== glls
2[[toc:]]
3glls (GL Lisp Shaders) lets you write [[https://www.opengl.org/documentation/glsl/|GLSL]] (OpenGL Shader Language) shaders in a convenient pseudo-scheme language in Chicken Scheme. The compilation into GLSL happens at compile-time for zero run-time cost. Run-time compilation and dynamic recompilation is also supported. To those that want to dynamically construct shaders: I solute you.
4
5In addition to the eponymous module, glls also provides the {{glls-render}} module. {{glls-render}} enhances glls to create automatic rendering functions for each pipeline. When compiled, these rendering functions are created in efficient C, although dynamic functions are also provided. See the section [[#automatic-render-functions|Automatic render functions]] for details.
6
7The idea for glls was hugely inspired by [[https://github.com/cbaggers/varjo|Varjo]]. Before learning about Varjo, I had never considered the possibility of writing shaders in anything but the GLSL. Seeing them being written in Lisp was a major, "Of course!" moment.
8
9That said, while this library bears some superficial resemblance to Varjo, the approach is quite different. While Varjo does a lot of work to validate the the lispy-glls expressions (including type checking), glls only performs cursory syntactic checking. The result of this is that one could probably write shaders in Varjo without knowing the GLSL and could be reasonably sure that those shaders would always compile to something that would mostly work. glls makes no such promises, so it is entirely possible to generate GLSL that won’t compile. Being able to understand GLSL code is therefore a prerequisite for successful shader debugging. The GLSL code output by glls is beautifully formatted, thanks to Alex Shinn’s amazing [[http://synthcode.com/scheme/fmt/|fmt]] library. fmt is responsible for far more than just the GLSL formatting, since it is basically a compiler of its own. The compilation portion of glsl is more or less a thin layer on top of fmt.
10
11
12=== Requirements
13* make
14* fmt
15* matchable
16* miscmacros
17* opengl-glew
18* srfi-42
19
20
21=== Documentation
22<parameter> glsl-version</parameter>
23
24The default GLSL version used by shaders. Defaults to {{330}}.
25
26
27==== Shaders
28<record> (shader TYPE SOURCE INPUTS OUTPUTS UNIFORMS PROGRAM)</record>
29
30Used to represent shaders. Returned by {{define-shader}} and {{create-shader}}. It should not typically be necessary to access the slots of this record.
31
32<macro> (define-shader SHADER-NAME GLLS-SHADER)</macro>
33
34Defines a new {{shader}} named {{NAME}}. The (unquoted) form {{GLLS-SHADER}} should conform to language defined in the section [[#the-glls-shader-language|The glls shader language]]. Before shaders are used, they must be compiled by OpenGL with {{compile-shader}}.
35
36<procedure> (create-shader GLLS-SHADER #!key INPUTS)</procedure>
37
38Creates (at run-time) a new {{shader}}. The form {{GLLS-SHADER}} should conform to language defined in the section [[#the-glls-shader-language|The glls shader language]]. The key {{INPUTS}} can be used to include additional inputs to the shader. Before shaders are used, they must be compiled by OpenGL with {{compile-shader}}.
39
40<procedure> (compile-glls GLLS-SHADER #!key INPUTS)</procedure>
41
42Returns the source string for a shader. The form {{GLLS-SHADER}} should conform to language defined in the section [[#the-glls-shader-language|The glls shader language]]. The key {{INPUTS}} can be used to include additional inputs to the shader.
43
44<procedure> (compile-shader SHADER)</procedure>
45
46Compile (in OpenGL) {{SHADER}}. Nothing is done if the shader has already been compiled. This typically does not need to be called, since {{compile-pipeline}} does so. Must be called while there is an active OpenGL context.
47
48
49==== Pipelines
50''Pipelines'' are the term that glsl uses to describe a collection of shaders that will be linked together. This is equivalent to a GL ''program'', just less ambiguously named.
51
52<record> (pipeline SHADERS ATTRIBUTES UNIFORMS PROGRAM)</record>
53
54Created with {{define-pipeline}} or {{create-pipeline}}, contains the data needed for a pipeline. {{SHADERS}} is the list of shader records. {{ATTRIBUTES}} and {{UNIFORMS}} are lists of the attributes and uniforms of the shader, specified as {{(name . type)}} pairs before compilation (with {{compile-pipeline}} or {{compile-pipelines}}) and {{(name location type)}} lists after compilation. {{PROGRAM}} is the GL ID of the program (always 0 before compilation).
55
56<macro> (define-pipeline PIPELINE-NAME . SHADERS)</macro>
57
58Defines a new {{pipeline}} named {{NAME}}. The {{SHADERS}} should either be forms conforming to language defined in the section [[#the-glls-shader-language|The glls shader language]], {{shader}}s defined by {{define-shader}}, or a mix of the two. Pipelines must have at least one vertex and one fragment shader to be able to compile. Before pipelines are used, they must be compiled by OpenGL with {{compile-pipeline}} or {{compile-pipelines}}.
59
60{{define-pipeline}} behaves differently when it is being evaluated ''and'' when a given pipeline is being redefined. In this case, the new pipeline inherits the GL program ID of the old one. Additionally, the pipeline is compiled by OpenGL right away (and as a consequence, so are any pipelines that are pending compilation). This is done so that pipelines can be edited and reevaluated in a REPL session and one’s scene will be updated as expected. See the [[https://github.com/AlexCharlton/glls/blob/master/examples/interactive.scm|interactive example]] for an example of how this can be accomplished.
61
62{{define-pipeline}} has additional effects when used with the {{glls-render}} module (see [[#automatic-render-functions|Automatic render functions]]).
63
64<procedure> (create-pipeline . SHADERS)</procedure>
65
66Creates (at run-time) a new {{pipeline}}. The {{SHADERS}} should either be forms conforming to language defined in the section [[#the-glls-shader-language|The glls shader language]], {{shader}}s, or a mix of the two. Pipelines must have at least one vertex and one fragment shader to be able to compile. Before pipelines are used, they must be compiled by OpenGL with {{compile-pipeline}} or {{compile-pipelines}}.
67
68<procedure> (compile-pipeline PIPELINE)</procedure>
69
70Compile (in OpenGL) the {{PIPELINE}} and sets its {{PROGRAM}} slot to the OpenGL program ID. If the pipeline’s {{PROGRAM}} slot is already set to a non-zero value, this ID will be reused for the new program. Compiles all of the pipeline’s shaders with {{compile-shader}}. Must be called while there is an active OpenGL context.
71
72<procedure> (compile-pipelines)</procedure>
73
74Compile (as per {{compile-pipeline}}) all the pipelines defined by {{define-pipeline}} and {{create-pipeline}}. Must be called while there is an active OpenGL context.
75
76<procedure> (pipeline-uniform UNIFORM PIPELINE)</procedure>
77
78Return the location of {{UNIFORM}}. The {{PIPELINE}} must be compiled before this function can be used.
79
80<procedure> (pipeline-attribute ATTRIBUTE PIPELINE)</procedure>
81
82Return the location of {{ATTRIBUTE}}. The {{PIPELINE}} must be compiled before this function can be used.
83
84
85==== The glls shader language
86
87===== Shader syntax
88The shaders of glls – the forms that {{define-shader}}, {{define-pipeline}}, etc. expect – have the following syntax:
89
90    (<type> [version: <version>] [extensions: <extension>] [pragmas: <pragma>])
91      <inputs> <body> -> <outputs>
92
93{{type}} is the keyword type of the shader. It must be one of {{#:vertex}}, {{#:fragment}}, {{#:geometry}}, {{#:tess-control}}, {{#:tess-evaluation}}, or {{#:compute}}.
94
95{{version}} is the integer version number of the shader, i.e. the number you would write at the top of the shader source (e.g. {{#version 410}}). Defaults to the {{glsl-version}} parameter.
96
97{{extensions}} is the list of GLSL extensions desired (in string form). E.g. {{'("GL_EXT_gpu_shader4 : enable")}}. Defaults to {{'()}}
98
99{{pragmas}} is the list of GLSL pragmas desired (in string form). E.g. {{'("optimize(on)")}}. Defaults to {{'()}}
100
101{{inputs}} is a list of the input variables to the shader. These are given in {{(name type)}} lists. The keyword {{uniform:}} may be used, and all following inputs will be uniforms. E.g.: {{((vertex #:vec2) (color #:vec3) uniform: (view-matrix #:mat4))}}
102
103{{body}} is the form representing the code of the shader. See the section [[#shader-lisp|Shader Lisp]] for an explanation of the kind of code that is expected.
104
105{{outputs}} is a list of the output variables from the shader. These are given in {{(name type)}} lists.
106
107
108===== Shader Lisp
109For the most part, the Lisp used to define glls shaders looks like Scheme with one notable difference: types must be specified whenever a variable or function is defined. Under the hood, forms are being passed to [[https://wiki.call-cc.org/eggref/4/fmt#c-as-s-expressions|fmt]], so everything that you can do there will work in glls. Details of the Lisp used for shaders is provided in the following sections.
110
111It should be possible to do almost anything in glls that you would want to do with the GLSL. Known exceptions to this is are: layout qualifiers (which I don’t feel are terribly relevant in the context of Scheme, at least not until uniform locations become prevalent), do-while loops (which have no Scheme analog), uniform blocks (for no good reason), {{#error}}, {{#line}}, {{#undef}}, and struct uniforms (implementation reasons). Let me know if there are any features that you find lacking.
112
113Keep in mind that glls cannot do anything that the GLSL can’t, such as making anonymous or recursive functions.
114
115
116====== Variables and naming
117Symbols in glls are transformed from a Scheme style into the C style used in the GLSL. Letters after dashes are uppercased (i.e., symbols become camelCased). Symbols prefixed by {{gl:}} in glls become prefixed by {{gl_}} in GLSL.
118
119For programmer-defined variables this has little consequence. The importance of learning the renaming conventions comes when you want to call GLSL functions or variables. Examples of mappings between glls and GLSL names are: {{gl:position}} → {{gl_Position}}, {{float-bits-to-uint}} → {{floatBitsToUint}}, {{shadow-2d-proj-lod}} → {{shadow2DProjLod}}, and {{sampler-2d-ms-array}} → {{sampler2DMSArray}}. Two special cases are {{emit-vertex}} and {{end-primitive}} which are translated into the functions {{EmitVertex}} and {{EndPrimitive}} respectively (which, for some reason, go against the usual GLSL naming conventions).
120
121
122====== Types
123When defining variables or functions in glls, types must be supplied. Basic types (e.g. {{int}}, {{mat2x2}}) are given either as a symbol or keyword (e.g. {{int}}, {{#:mat2x2}}), whichever is preferred. Types with qualifiers (e.g. {{lowp float}}, {{out mediump vec2}}) are given as lists (e.g. {{(lowp float)}}, {{(out mediump vec2)}}).
124
125Arrays are specified as lists beginning with the keyword {{#:array}}. The next element in the list is the type, while the optional third element is the size. E.g. {{(#:array int 5)}}. When used with qualifiers, the array takes the place of the type, e.g. {{(highp (#:array float))}}.
126
127
128====== Functions
129GLSL functions and operators are all called like normal Lisp functions. In almost all cases the GLSL symbol (taking into account the renaming described in [[#variables-and-naming|Variables and naming]] can be used, while many operators can be called with their Scheme counterpart. The only operators that may not be used directly are {{|}}, {{||}}, {{|=}}, {{.}}, {{=}}, and array reference which must be called with their counterparts.
130
131The following is a mapping between glls aliases for GLSL functions and operators:
132
133* {{modulo}}: {{%}}
134* {{expt}}: {{pow}}
135* {{equal?}}, {{eqv?}}, {{eq}}, {{=}}: {{==}}
136* {{set!}}: {{=}}
137* {{and}}: {{&&}}
138* {{or}}: {{||}}
139* {{not}}: {{!}}
140* {{bitwise-and}}, {{bit-or}}: {{&}}
141* {{bitwise-ior}}, {{bit-or}}: {{|}}
142* {{bitwise-xor}}, {{bit-xor}}: {{^}}
143* {{bitwise-not}}, {{bit-not}}: {{~}}
144* {{arithmetic-shift}}: {{<<}}
145* {{field}}: {{.}} (struct field reference, e.g. {{(field point x)}} → {{point.x}})
146* {{swizzle}}: {{.}} (vector swizzling, e.g. {{(swizzle color r g)}} → {{color.rg}})
147* {{array-ref}}, {{vector-ref}}: {{[]}} (array reference, e.g. {{(array-ref a 4)}} → {{a[4]}})
148* {{length}}: {{.length()}} (vector length, e.g. {{(length vec)}} → {{vec.length()}})
149
150
151====== Definition
152Variables, functions, and records (structs) are defined much like they are in Scheme, with the additional requirement of including types.
153
154    (define <name> <type> [<value>])
155
156Defines the variable {{name}}. When {{type}} is an array, a vector literal (eg. {{#(1 2 3)}}) may be used.
157
158    (define (<name> [(<parameter> <type>) ...]) <return-type> <body> ...)
159
160Defines the function {{name}}. The last expression in the body of a non-void function is automatically returned.
161
162    (let ((<name> <type> [<value>]) ...) <body> ...)
163
164Defines the supplied variables. When {{type}} is an array, a vector literal (eg. {{#(1 2 3)}}) may be used. Note that, unlike Scheme, the variables created will continue to exist outside of the {{let}} (until the extent of whatever lexical scope the {{let}} exists within). In other words, {{let}} does not introduce scope. Note also that variables defined in {{let}} are within the scope of variables that are subsequently defined in the same {{let}} (i.e. {{let}} functions like {{let*}} in Scheme, and in fact {{let*}} may be used if preferred).
165
166    (define-record <name> (<type> <field>) ...)
167
168Defines the struct {{name}}.
169
170
171====== Control
172The following can be used with identical syntax to scheme:
173
174    (if <test> <true> [<false>])
175   
176    (cond (<test> <result> ...) ... (else <result>))
177   
178    (case <key> ((<value> ...) <result> ...) ... (else <result>))
179   
180    (begin <body> ...)
181
182Keep in mind that they may only be used in the same place as their corresponding GLSL statements, with the exception of {{begin}}, which can only be used where it is possible to have multiple expressions.
183
184
185====== Iteration
186    (for <init> <condition> <update> <body> ...)
187
188GLSL style {{for}} loop.
189
190    (do-times (<var> [<start>] <end>) <body> ...)
191
192Equivalent to {{(for (define <var> #:int <start>) (< <var> <end>) (++ <var>) <body> ...)}}. {{start}} defaults to 0.
193
194    (while <condition> <body> ...)
195
196GLSL style {{while}} loop.
197
198
199====== Jumps
200All GLSL jumps ({{continue}}, {{break}}, {{return}}, {{discard}}) can be called like functions. Return may accept one argument. Keep in mind that the last expression in a non void function is automatically returned.
201
202
203====== Pre-processor
204The following forms can be used to add pre-processor directives:
205
206    (%define <name> [<value>])
207   
208    (%if <test> <true> [<false>])
209   
210    (%ifdef <value> <true> [<false>])
211   
212    (%ifndef <value> <true> [<false>])
213
214
215==== Automatic render functions
216By using the {{glls-render}} module, you can have glls automatically generate a function that will render an object with your glls shader. {{glls-render}} exports a new {{define-pipeline}} that defines a set of functions used for rendering and managing the objects that will be rendered. {{glls-render}} should not be used with the {{glls}} module: It reexports everything that you need from {{glls}}.
217
218Recalling {{define-pipeline}}:
219
220    (define-pipeline PIPELINE-NAME . SHADERS)
221
222There is one difference that you need to know when calling {{glls-render}}’s {{define-pipeline}}: All shaders must include a list of the shader’s uniforms since the uniforms are the important information needed to derive rendering functions. This means that if you previously define some shaders (for example: {{my-vertex-shader}} and {{my-fragment-shader}}) and you wish to combine them in a pipeline, you ''must'' include the uniforms in the pipeline definition. This is done with a list that takes the form {{(SHADER uniform: [UNIFORM] ...)}}. This list must be present even if the shader does not use any uniforms. For example:
223
224    (define-pipeline my-pipeline
225      (my-vertex-shader uniform: mvp-matrix inverse-transpose-matrix)
226      (my-fragment-shader uniform:))
227
228Of course, if you are defining the shaders in the pipeline, then a separate list of uniforms is not necessary.
229
230{{glls-render}} causes {{define-pipeline}} to define several new functions. First is {{render-PIPELINE-NAME}}. {{render-PIPELINE-NAME}} takes one argument: a renderable object (see [[#renderables|Renderables]]).
231
232The {{render-PIPELINE-NAME}} function works differently depending on whether the {{define-pipeline}} has been compiled or interpreted (although the end results should stay the same). When {{define-pipeline}} is compiled, the resulting {{render-PIPELINE-NAME}} function is compiled directly to efficient (non-branching) C. When {{define-pipeline}} is interpreted, {{render-PIPELINE-NAME}} calls a generic rendering function that is not nearly as fast.
233
234
235===== Renderables
236In order to use one of the automatically generated render functions, you must have something to render. That’s why {{define-pipeline}} also defines a function that constructs a renderable object: {{make-SHADER-NAME-renderable}}. This function takes a number of keyword arguments:
237
238* {{vao:}} – A VAO such as those returned by [[http://api.call-cc.org/doc/opengl-glew/make-vao|opengl-glew’s {{make-vao}}]]. I.e.: A VAO that binds an array of attributes – for each element in the pipeline – as well as an element array.
239* {{mode:}} – The drawing mode to use when drawing the elements of the VAO. Must be one of (opengl-glew’s): {{+points+}}, {{+line-strip+}}, {{+line-loop+}}, {{+lines+}}, {{+line-strip-adjacency+}}, {{+triangles+}}, {{+triangle-strip+}}, {{+triangle-fan+}}, {{+triangles-adjacency+}}, {{+triangle-strip-adjacency+}}, or {{+patches+}}. Defaults to {{+triangles+}}.
240* {{n-elements:}} – The number of elements (vertices) to draw.
241* {{element-type:}} – The type of the values in the VAO’s element array. Must be one of {{+unsigned-byte+}}, {{+unsigned-short+}}, or {{+unsigned-int+}}.
242* {{offset:}} – A byte offset to the location of the desired indices to draw.
243* {{data:}} – An optional pointer to an appropriate glls renderable object. If not provided, a fresh renderable object will be created. [[https://github.com/AlexCharlton/glls/blob/master/gllsRender.h|gllsRenderable.h]] defines the structs used for renderables. Which struct is used for a given pipeline is chosen based on the number of uniforms present in the pipeline.
244
245See the [[https://www.opengl.org/sdk/docs/man/html/glDrawElements.xhtml|{{glDrawElements}} documentation]] for more information about these expected arguments.
246
247{{make-SHADER-NAME-renderable}} also expects one keyword argument for each uniform in the pipeline. These arguments should either be an f32vector, an s32vector, a u32vector, a pointer to the uniform data, or – in the case of a texture – a fixnum. Even if the uniform is a single value (e.g. a float), it must still be passed as a vector (or a pointer). This lets the value of the uniform be updated independently of the renderable.
248
249Additionally, {{define-pipeline}} defines a number of renderable setters for each of the keyword arguments accepted by {{make-SHADER-NAME-renderable}}. These are named:
250
251* {{set-SHADER-NAME-renderable-vao!}}
252* {{set-SHADER-NAME-renderable-mode!}}
253* {{set-SHADER-NAME-renderable-n-elements!}}
254* {{set-SHADER-NAME-renderable-element-type!}}
255* {{set-SHADER-NAME-renderable-offset!}}
256
257And for each uniform in the pipeline, {{set-SHADER-NAME-renderable-UNIFORM-NAME!}} is created.
258
259
260===== Fast render functions
261When compiled, the render function defined by {{define-pipeline}} is actually a combination of three “fast” render functions: a begin render function, a render function, and an end render function. This is done so that, if desired, all of the renderables that belong to the same pipeline may be rendered at the same time, without needing to perform expensive calls like program changes or texture binding more than once. To use these functions, simply call the begin render function with the first renderable, then call the render function on all renderables (including the first), finally calling the end render function (with no arguments) to clean up.
262
263{{define-pipeline}} does not define all of these functions separately, but instead defines a single function with which to access them: {{PIPELINE-NAME-fast-render-functions}}. This function returns six values: the begin render function, the render function, the end render function, and pointers to those same C functions in that order.
264
265One major assumption must be kept in mind while working with the fast render functions: textures are only bound once. In other words: it is assumed that that all of the renderables belonging to the same pipeline share a common “sprite sheet” (or other shared texture type). If this assumption does not hold true, simply use the standard render function, or call the begin render function for every set of renderables that uses a separate texture.
266
267
268=== Examples
269These examples depends on the [[http://wiki.call-cc.org/eggref/4/glfw3|glfw3]] egg for window and context creation. The examples presented here illustrate only very basic shader definition and loading. For more complete examples, see the [[https://github.com/AlexCharlton/glls/tree/master/examples|examples directory]] of the source.
270
271Aside from knowing how to write glls shaders, only one macro, one function, and one record is necessary to use glls: {{define-pipeline}}, {{compile-pipelines}}, and the record {{pipeline}}. This example illustrates this minimal pipeline creation
272
273<enscript highlight="scheme">    
274(import chicken scheme)
275
276(use glls (prefix glfw3 glfw:) (prefix opengl-glew gl:))
277
278(define-pipeline foo
279  ((#:vertex) ((vertex #:vec2) (color #:vec3) uniform: (mvp #:mat4))
280     (define (main) #:void
281       (set! gl:position (* mvp (vec4 vertex 0.0 1.0)))
282       (set! c color))
283     -> ((c #:vec3)))
284  ((#:fragment) ((c #:vec3))
285     (define (main) #:void
286       (set! frag-color (vec4 c 1.0)))
287     -> ((frag-color #:vec4))))
288
289(glfw:with-window (640 480 "Example" resizable: #f)
290   (gl:init)
291   (compile-pipelines)
292   (print foo)
293   (gl:use-program (pipeline-program foo)))
294</enscript>
295
296This example is similar to the first, but also illustrates the ability to define pipelines in different ways.
297
298<enscript highlight="scheme">    
299(import chicken scheme)
300
301(use glls (prefix glfw3 glfw:) (prefix opengl-glew gl:))
302
303(define-pipeline foo
304  ((#:vertex) ((vertex #:vec2) (color #:vec3) uniform: (mvp #:mat4))
305     (define (main) #:void
306       (set! gl:position (* mvp (vec4 vertex 0.0 1.0)))
307       (set! c color))
308     -> ((c #:vec3)))
309  ((#:fragment) ((c #:vec3))
310     (define (main) #:void
311       (set! frag-color (vec4 c 1.0)))
312     -> ((frag-color #:vec4))))
313
314(define-shader bar (#:vertex)
315    ((vertex #:vec2) (color #:vec3) uniform: (mvp #:mat4))
316  (define (main) #:void
317    (set! gl:position (* mvp (vec4 vertex 0.0 1.0)))
318    (set! c color))
319  -> ((c #:vec3)))
320
321(define-pipeline baz
322  bar
323  (cadr (pipeline-shaders foo)))
324
325(glfw:with-window (640 480 "Example" resizable: #f)
326   (gl:init)
327   (compile-pipelines)
328   (print foo)
329   (print baz))
330</enscript>
331
332
333=== Version history
334
335==== Version 0.3.0
33630 May 2014
337
338* Support dynamic reevaluation of pipelines
339
340
341==== Version 0.2.2
34229 May 2014
343
344* Add make as a dependency
345
346'''Version 0.2.1'''
347
348* Fix bug affecting dynamic use of glls-render
349
350'''Version 0.2.0'''
351
35228 May 2014
353
354* Automatic render function generation
355* Removed {{eval}} from {{defpipeline}} (which broke some things when used in modules)
356* Renamed {{defpipeline}}, {{defshader}}
357
358
359==== Version 0.1.0
360* Initial release
361
362
363=== Source repository
364Source available on [[https://github.com/AlexCharlton/glls|GitHub]].
365
366Bug reports and patches welcome! Bugs can be reported via GitHub or to alex.n.charlton at gmail.
367
368
369=== Author
370Alex Charlton
371
372
373=== Licence
374BSD
Note: See TracBrowser for help on using the repository browser.