Making a DSL for Generating Images
Very often when trying to work on computer vision problems, the lack of sufficient data becomes a big issue. This is especially true when working with neural networks.
Would it not be great if we could have a limitless source of new and original data?
This thought has led me to create a Domain Specific Language that allows the creation of images in various configurations. These images can then be used for training and testing machine learning models. Now, as the name suggests, the images DSL generate can usually be used only in a narrow domain.
Requirements
For my particular case, I focus on object detection. The compiler of the language must generate images that fulfill the following criteria:
- images contain various shapes (think emoji).
- the number and position of individual shapes are configurable.
- size of the image and of the shapes is configurable.
The language itself must be as simple as possible. First I want to be able to define the size of the output image and then the size of the shapes. After that, I want to express the actual configuration of the image. To make things easier, I think of the image as being a table, so each shape can go in a cell. Each new row starts from the left and then it gets filled with shapes.
Implementation
To build my DSL I have chosen to use a combination of ANTLR, Kotlin and Gradle. ANTLR is a parser generator. Kotlin is a JVM based language similar to Scala. Gradle is build system similar to sbt.
Prerequisites
To follow the tutorial, you will need to have Java 1.8 and Gradle 4.6.
Initial set-up
Create a folder that will contain the DSL.
> mkdir shaperdsl
> cd shaperdsl
Create the build.gradle file. This file is used to list the dependencies of the project and to configure additional Gradle tasks. If you want to reuse this file you should typically modify only the namespaces and the main class.
> touch build.gradle
Here is the content of the file
buildscript {
ext.kotlin_version = '1.2.21'
ext.antlr_version = '4.7.1'
ext.slf4j_version = '1.7.25'
repositories {
mavenCentral()
maven {
name 'JFrog OSS snapshot repo'
url 'https://oss.jfrog.org/oss-snapshot-local/'
}
jcenter()
}
dependencies {
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath 'com.github.jengelman.gradle.plugins:shadow:2.0.1'
}
}
apply plugin: 'kotlin'
apply plugin: 'java'
apply plugin: 'antlr'
apply plugin: 'com.github.johnrengelman.shadow'
repositories {
mavenLocal()
mavenCentral()
jcenter()
}
dependencies {
antlr "org.antlr:antlr4:$antlr_version"
compile "org.antlr:antlr4-runtime:$antlr_version"
compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
compile "org.jetbrains.kotlin:kotlin-reflect:$kotlin_version"
compile "org.apache.commons:commons-io:1.3.2"
compile "org.slf4j:slf4j-api:$slf4j_version"
compile "org.slf4j:slf4j-simple:$slf4j_version"
compile "com.audienceproject:simple-arguments_2.12:1.0.1"
}
generateGrammarSource {
maxHeapSize = "64m"
arguments += ['-package', 'com.example.shaperdsl']
outputDirectory = new File("build/generated-src/antlr/main/com/example/shaperdsl".toString())
}
compileJava.dependsOn generateGrammarSource
jar {
manifest {
attributes "Main-Class": "com.example.shaperdsl.compiler.Shaper2Image"
}
from {
configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
}
}
task customFatJar(type: Jar) {
manifest {
attributes 'Main-Class': 'com.example.shaperdsl.compiler.Shaper2Image'
}
baseName = 'shaperdsl'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
The language parser
The parser is built as ANTLR grammar.
mkdir -p src/main/antlr
touch src/main/antlr/ShaperDSL.g4
with the following content:
grammar ShaperDSL;
shaper : 'img_dim:' img_dim ',shp_dim:' shp_dim '>>>' ( row ROW_SEP)* row '<<<' NEWLINE* EOF;
row : ( shape COL_SEP )* shape ;
shape : 'square' | 'circle' | 'triangle';
img_dim : NUM ;
shp_dim : NUM ;
NUM : [1-9]+ [0-9]* ;
ROW_SEP : '|' ;
COL_SEP : ',' ;
NEWLINE : '\r\n' | 'r' | '\n';
You can see how the language internals now become clear. To generate the grammar source code run:
> gradle generateGrammarSource
You will end up with the generated code in build/generated-src/antlr.
> ls build/generated-src/antlr/main/com/example/shaperdsl/
ShaperDSL.interp ShaperDSL.tokens ShaperDSLBaseListener.java ShaperDSLLexer.interp ShaperDSLLexer.java ShaperDSLLexer.tokens ShaperDSLListener.java ShaperDSLParser.java
Abstract Syntax Tree
The parser transforms the source code into a tree of objects. The tree of objects is what the compiler uses as the data source. To obtain the AST, a meta model of the tree needs to be defined first.
> mkdir -p src/main/kotlin/com/example/shaperdsl/ast
> touch src/main/kotlin/com/example/shaper/ast/MetaModel.kt
MetaModel.kt contains the definitions for the classes of objects used in the language, starting with the root. They all inherit from a Node interface. The tree hierarchy is visible in the class definition.
package com.example.shaperdsl.ast
interface Node
data class Shaper(val img_dim: Int, val shp_dim: Int, val rows: List<Row>): Node
data class Row(val shapes: List<Shape>): Node
data class Shape(val type: String): Node
Next is the mapping to the AST
> touch src/main/kotlin/com/example/shaper/ast/Mapping.kt
Mapping.kt is used to build the AST by using the classes defined in the MetaModel.kt along with the input from the parser.
package com.example.shaperdsl.ast
import com.example.shaperdsl.ShaperDSLParser
fun ShaperDSLParser.ShaperContext.toAst(): Shaper = Shaper(this.img_dim().text.toInt(), this.shp_dim().text.toInt(), this.row().map { it.toAst() })
fun ShaperDSLParser.RowContext.toAst(): Row = Row(this.shape().map { it.toAst() })
fun ShaperDSLParser.ShapeContext.toAst(): Shape = Shape(text)
Here is a a graphical representation of an AST:
img_dim:100,shp_dim:8>>>square,square|circle|triangle,circle,square<<<
translates to:
The compiler
The compiler is the last part. It uses the AST to create a concrete representation in the desired format, in this case, an image.
> mkdir -p src/main/kotlin/com/example/shaperdsl/compiler
> touch src/main/kotlin/com/example/shaper/compiler/Shaper2Image.kt
There is a lot cramped down in this file. I will try to break it down.
ShaperParserFacade is a wrapper on top of ShaperAntlrParserFacade which constructs the actual AST from the source code provided. Shaper2Image is the main compiler class. After it receives the AST from the parser, it goes through the all the objects inside it and creates graphical objects that it then inserts them inside a container image. It then finally returns the binary representation of the image. A main function in the class’ companion is provided as well to allow testing.
package com.example.shaperdsl.compiler
import com.audienceproject.util.cli.Arguments
import com.example.shaperdsl.ShaperDSLLexer
import com.example.shaperdsl.ShaperDSLParser
import com.example.shaperdsl.ast.Shaper
import com.example.shaperdsl.ast.toAst
import org.antlr.v4.runtime.CharStreams
import org.antlr.v4.runtime.CommonTokenStream
import org.antlr.v4.runtime.TokenStream
import java.awt.Color
import java.awt.image.BufferedImage
import java.io.ByteArrayInputStream
import java.io.ByteArrayOutputStream
import java.io.File
import java.io.InputStream
import javax.imageio.ImageIO
object ShaperParserFacade {
fun parse(inputStream: InputStream) : Shaper {
val lexer = ShaperDSLLexer(CharStreams.fromStream(inputStream))
val parser = ShaperDSLParser(CommonTokenStream(lexer) as TokenStream)
val antlrParsingResult = parser.shaper()
return antlrParsingResult.toAst()
}
}
class Shaper2Image {
fun compile(input: InputStream): ByteArray {
val root = ShaperParserFacade.parse(input)
val img_dim = root.img_dim
val shp_dim = root.shp_dim
val bufferedImage = BufferedImage(img_dim, img_dim, BufferedImage.TYPE_INT_RGB)
val g2d = bufferedImage.createGraphics()
g2d.color = Color.white
g2d.fillRect(0, 0, img_dim, img_dim)
g2d.color = Color.black
var j = 0
root.rows.forEach{
var i = 0
it.shapes.forEach {
when(it.type) {
"square" -> {
g2d.fillRect(i * (shp_dim + 1), j * (shp_dim + 1), shp_dim, shp_dim)
}
"circle" -> {
g2d.fillOval(i * (shp_dim + 1), j * (shp_dim + 1), shp_dim, shp_dim)
}
"triangle" -> {
val x = intArrayOf(i * (shp_dim + 1), i * (shp_dim + 1) + shp_dim / 2, i * (shp_dim + 1) + shp_dim)
val y = intArrayOf(j * (shp_dim + 1) + shp_dim, j * (shp_dim + 1), j * (shp_dim + 1) + shp_dim)
g2d.fillPolygon(x, y, 3)
}
}
i++
}
j++
}
g2d.dispose()
val baos = ByteArrayOutputStream()
ImageIO.write(bufferedImage, "png", baos)
baos.flush()
val imageInByte = baos.toByteArray()
baos.close()
return imageInByte
}
companion object {
@JvmStatic
fun main(args: Array<String>) {
val arguments = Arguments(args)
val code = ByteArrayInputStream(arguments.arguments()["source-code"].get().get().toByteArray())
val res = Shaper2Image().compile(code)
val img = ImageIO.read(ByteArrayInputStream(res))
val outputfile = File(arguments.arguments()["out-filename"].get().get())
ImageIO.write(img, "png", outputfile)
}
}
}
Now that everything is ready, the project can be built. An uber jar is created.
> gradle shadowJar
> ls build/libs
shaper-dsl-all.jar
Testing
All we have to do now is test if everything works, so try feeding some code like this:
> java -cp build/libs/shaper-dsl-all.jar com.example.shaperdsl.compiler.Shaper2Image \
--source-code "img_dim:100,shp_dim:8>>>circle,square,square,triangle,triangle|triangle,circle|square,circle,triangle,square|circle,circle,circle|triangle<<<" \
--out-filename test.png
A .png file is created that will look like this:
Conclusion
This is a simple DSL, it is not hardened and will probably break if used outside the way it is intended. However, it fits my purpose very well and I can use it to generate any desired number of unique sample images. It can easily be extended to be more configurable and can be used as a template for other DSL.
A complete example of the DSL can be found in my GitHub repository: https://github.com/cosmincatalin/shaper.
Comments