Quick Link to Deliverables | Docs
Project Set Up
This is a team project. Accept the assignment. Join your team. If you don't see your team's name (see the email), create a new team. After the team's repository is created, copy the link to clone your repository.
Import the project and fix the project's classpath to include the JUnit library as we have done several times now. There will still be compilation errors in the JUnit tests because I created some example tests, but you haven't written the code yet for them to work. (A little TDD.)
Now, set up something that will make things a little easier later.
Go to Preferences. In the search bar, start typing "file". One of
the first results should be "File Associations". Click on that.
Click the top Add button and add .exp
as a file type.
Then, add the associated editor for the .exp
files.
Search for "text" and then select "Text Editor". Repeat that process
with .conf
files.
Specifications for the "Core" Picasso Application
We are going to create an application that allows the user to create expressions that evaluate to colors and then eventually to images. This may sound a little strange, but the results can be quite spectacular.
Here is the general idea: an image is really an expression that maps points (x,y) to colors (r,g,b). The application evaluates the expression for each pixel (x-, y-point) in the image and stores the resulting color.
For this application, a color represents three real numbers, one each for the red, green and blue component, where the component values range from -1 to 1. For example, [-1,-1,-1] is black, [1,-1,-1] is red, and [1, 1,-1] is yellow. During computation of an expression, the value of each component should not be restricted to this range, but the final result of the expression should be clamped to within the range -1 to 1.
By default, like the colors, the domain of the image over X and Y
is from -1 to 1. The upper left corner of the image will be (-1,-1)
and the lower right corner will be (1, 1). Thus, the size of the image
will need to be mapped to the domain (in this case, [-1, 1]). Before
the expression is evaluated for each pixel, the
variables x
and y
should be set to the
current point to be evaluated.
Your application should allow a user to input expressions interactively and from a file and display the resulting image. Your application should display errors in a user-friendly way.
The Picasso Language
The syntax of the allowed expressions is given below.
Note: If a function is scalar, i.e., typically operates on a single value (e.g., sin(x)), then it should be applied to each of the color components in turn.
Expression | Syntax | Semantics | Examples |
---|---|---|---|
Constant | <any real number> [-]?[0-1]+(.[0-9]+)? |
a real valued number (note, to avoid potential ambiguity in parsing there should not be a space between the negative sign and the value) | .87 1 -0.4 |
Color |
[ constant, constant, constant ] |
an RGB color where each value can be any constant |
[1, -1, 0.25] [0.5, 1, 1] |
String | <any non-quote character between quotes> |
an image of the given file name is read.
This should return the nearest color from the image at the current (x, y) values If the image cannot be read, an error is thrown. |
"foo.jpg" "images/mickey.gif" |
Variable |
<any alpha-numeric string> [a-zA-z]+[a-zA-Z0-9]* |
an expression represented by a word
The two variables x and y should
always be defined to be the current coordinate in the image
domain
|
a
bugs q45 |
Assignment | var = expr | assigns an expression to a variable |
a = 1.0 bugs = "foo.png" a = bugs |
Unary Operator | op expr | prefixes an expression
! // negate (i.e., invert) a color |
!a !(t + a * 0.1) |
Binary Operator | expr op expr | combines two expressions into a single
expression (in precedence order)
^ // exponentiate * // times / // divide % // mod + // plus - // minus |
a + b
a / .2 a + 1.0 * c |
Unary Functions | fun(expr) | a function that takes an expression as its single argument
floor // round down ceil // round up abs // absolute value clamp // clamp results to [-1, 1] (e.g., 1.5 -> 1) wrap // wrap results around [-1, 1] (e.g., 1.5 -> -0.5) sin // sine cos // cosine tan // tangent atan // arc tangent exp // e ^ parameter log // log -- use the absolute value of the parameter rgbToYCrCb // convert color from RBG to luminance / chrominance space yCrCbToRGB // convert color to RGB from luminance / chrominance space |
sin(a * b)
abs(x) - y / 0.4 |
Multi-Argument Functions | fun(expr,...) | a function that takes two or more expressions as its arguments
perlinColor(expr, expr) // create random color based on 2D noise perlinBW(expr, expr) // create grey scale color based on 2D noise // imports image, tiling it so it may be repeated imageWrap(string, x-coord-expr, y-coord-expr) // imports image, clipping it so it only appears once imageClip(string, x-coord-expr, y-coord-expr) // If you look at the example on the Intrinsics page, it includes an // arbitrary expression for x and y, so those may go out of the -1 to 1 // bounds you are using. Thus clip and wrap are two different // ways to guarantee those values passed to the image's f(x, y) function // are within the bounds. |
perlinColor(x, y) perlinBW(y, x+x) |
No-Argument Function | function() | a function that takes no arguments
random() // returns a random color (the same color, no matter what value of x and y are used to evaluate it, each time it is called) |
random() |
Parentheses | (expr) | raises an expression's precedence | (a + b) * 0.3 !(bugs - 0.1) |
Single-line comments | // Comment | Everything after the // on the same line is ignored (like in C and Java) | // Example comment |
Operators have the following precedence (listed from highest to lowest):
() | parentheses |
! | unary operators |
^ | exponentiation |
*, /, % | multiplicative operators |
+, - | additive operators |
= | assignment |
For example, here are several images generated from basic expressions (intrinsics).
Note, not all of these functions are defined continuously. You should have appropriate error handling (e.g., divide-by-zero should silently return zero).
Evaluating Expressions
Expression Trees are trees in which the terminal nodes (leaves) represent variables or constants in an expression and the non-terminal (internal) nodes represent the operators. The inherent hierarchical nature of trees precisely capture operator precedence and provide a convenient mechanism for storing, traversing, and transforming expressions. Note, that unlike binary trees, each node of an Expression Trees can have any number of children, easily modeling operators with one, two, or even arbitrary numbers of arguments.
Expression Trees were first introduced in 1967 with the programming language LISP, in which the program itself is stored as an expression tree, called an s-expression, that can even be modified while the program is running! Since then, expression trees have formed the heart of many applications: representing genes, regular expressions, language grammars, and queries. They are so widespread that the World Wide Web Consortium (W3C) decided to use them as the core of the XML Document Object Model (DOM) specification, more than thirty years later!
The trickiest part about using expression trees is actually creating them from a given formatted input string. Parsing is the process of taking a string organized by a grammatical structure (that defines key characters to act as separators between the characters that are important, e.g., whitespace or semi-colon characters in Java) and producing a list of tokens in an order that is easy to convert into an expression tree. During the parsing process, the important sequences of characters are converted into token objects, while the separators are typically thrown out. If the input string contains grammatical errors such that it cannot be parsed, then an error is reported.
Here is a standard algorithm to convert an infix expression into a postfix expression, which is then easy to convert into an Expression Tree.
Testing
Your application should be thoroughly tested to prove that your confidence in it is justified. You should include whatever data files, driver programs, or unit tests you have used (as well as documentation on how to use them, if appropriate).
While the description of testing comes before the extensions, you should still test the extensions.
Extensions to the Picasso Functionality
There are many extensions to the core specifications possible. The most important thing is that your program is designed well (i.e., there is a clear separation between the syntax and semantics of the expression language and that it is clearly possible to change by adding only O(1) line to your existing code). This means your design should be open to adding new kinds of expressions while closed to changing the evaluation and parsing code. The requirements above and suggestions below are intended to help you to realize such a design.
The extensions below are intended to stretch your design further and to differentiate your program from others to capture the algorithmic art market. Your team must extend the program beyond the core specifications given above for full credit. These extensions must further the good design of your program and not simply be hacks of code added at the last minute. If you do not have time to implement an extension, partial credit may be given for excellent justification of how your design either supports adding such a feature already or how it would need to changed sufficiently to support such a feature. Your design should support adding any of these features reasonably easily.
If you are a four-person team, your team will implement three extensions. If you are a five-person team, your team will implement four extensions. If your team is considering a different extension than those listed, consult with me.
- Aside: it may seem like adding one person to the team "only" adds one more extension to the requirements. However, adding a team member also adds more management/collaboration/scheduling/coordination challenges throughout the process.
Generating expressions automatically
For all the following, also consider how you will allow users to trigger the generation and what input, if any, you need from the user.
- easily combine saved expressions into new expressions
- generate expressions randomly - your code should have a chance to make [almost] any possible expression in the Picasso language.
- generate expressions based on a string
- "breed" new expressions by combining old expressions
- allow the user other ways to control or influence the probability of generating terms in a random expression
How expressions are used to generate images (rather than simply evaluating them for each pixel):
- iterate the expression to
create fractals.
- Here is a tutorial about the Mandelbrot Set in Python that could serve as a good starting point.
- If you implement this extension, it is a good idea to also implement the zoom extension in the GUI below.
- solve for the roots of the expression
- animate an expression over time rather than simply creating a single image
- a time parameter that varies from zero to one (or any range) turns any expression into an animation (if it goes zero to one and back to zero, then it can be looped)
- if breeding is implemented as well, the animation can be the "genetic cross-dissolves" between the parent and the child
Improving Usability of GUI
For each of these, consider what will be the most intuitive for the users. Draw out the interaction on paper first. Have your teammates try out the extension to see if it can be approved. What kind of feedback (e.g., messages displayed in the GUI) would be helpful to the user?
- display the current defined variable names and their values (and perhaps thumbnail sized images)
- allow users to save a history (or a favorites list or rankings, perhaps "pinning" them) of old expressions as well as view or evaluate the history.
- allow users to view multiple images at once (either in separate windows, tabs, or a grid of thumbnails)
- allow users to change the size of the image displayed as a result of evaluating the expression
- allow users to zoom in or out on a part of the image (note, this changes the (x, y) domain used to compute the image)
- allow users to "debug" expressions by using the mouse to display the point and evaluated values at that point
- allow users to refer to history of expressions in this session by entering their history number prefixed by "$" (i.e., "$3") (It's helpful if the user can view their history)
- allow users to use the up and down arrows to move through the history of expressions
Making the evaluation code more general and efficient:
- prune the expression trees based on useless arithmetic branches (e.g., multiplication by 1 or 0)
- recognize identical sub-expressions to avoid recalculating them when necessary
- recognize sub-expressions that do not depend on either x or y so they can be computed only once per image
- recognize common sub-expressions within saved favorites so they can be given a higher probability when generating new expressions
Resources
- Artificial Evolution for Computer Graphics by Karl Sims (the original paper and inspiration for this project)
- Manipulating Parameters for Algorithmic Image Generation by Mike King
Picasso Documentation
Your Teams' Javadocs - Updated daily at 3:58 a.m.:
- Code Catalysts' Javadocs
- Dynamos' Javadocs
- Game Changers' Javadocs
- Hot Shots' Javadocs
- Invincibles' Javadocs
- Phenoms' Javadocs
- Visionaries' Javadocs
- Wizards' Javadocs
Deliverables
This project is worth 25% of your final course grade.
Preparation: analysis of given code, planning (10%) - Individual
Preparation Assignment Specification
Due Friday, Nov 15 before class
Preliminary functionality (15%) - Team
Due Friday, Nov 22 before class
This version should at least:
- allow the user to input an expression interactively that
includes at least one binary operator and 4-5 functions and display an
image from the resulting expression.
Each team member should implement and commit at least one of the unary functions. However, your team should not implement more than about 6. You don't want that much breadth yet because you may realize that, if you refactor, you'll have less work to do as you add new ones.
The text box should be wide enough for longer expressions that the application will be able to handle for the final submission.
After the user inputs the expression, the user should be able to hit enter or click the "Evaluate" button, and, in both cases, the image generated from the expression will be displayed.
- have unit tests of the binary operator and your implemented functions. All your JUnit tests should compile, run, and pass. (If you're using a test-driven development approach--which I encourage!, you can put those tests in a separate, marked file or just comment out the ones that aren't passing for this deliverable.)
- have your team name in the title bar of the GUI.
- list all team members' names in the
README.md
file, near the top, in a nice way when viewed in GitHub
More credit will be given to an application that gets this basic functionality working very well rather than trying to implement parts of the specification partially.
Create
a new release of the application. Tag it as version
v0.3
and title it as Preliminary
Functionality
. The target should be the main
branch. Now, you'll be able to easily refer back to this version.
While the given code base had compiler errors, your submitted code
should not (unless you have classes specifically for test-driven
development).
The team will demo this application in class to the professor and discuss their design decisions. You will also state what extensions you are planning to implement for the final application.
Ungraded Objectives. Think about
- your team's collaboration; what is going well? what should change? What problems using git need to be resolved?
- what you need to complete for the final implementation. With your current design, how well does your design extend for the next steps, including the extensions? What could be designed better? Should you refactor anything so that adding new code will be easier? An hour of thinking about the design and changing the code to improve the design will be worth hours of time later.
Intermediate Functionality (15%) - Team
Due: Fri, Dec 6 before class
This version should at least
- handle (simple) assignments, e.g.,
a = x binop y
and then if a user entersa
, the user sees the same as they would the right-hand side of the assignment (includes testing)Note: You should not do any "special handling" for assignment statements, i.e., it's part of the Picasso language specification, so the Picasso language interpreter should be able to process it. You should integrate this into the existing framework. Consider why it is problematic for assignment statements to not be handled by the interpreter from a design perspective.
- implement one image-manipulation function completely
(i.e.,
imageWrap
orimageClip
includes testing). Note that implementing String and wrap and/or clamp will be useful to implementing this objective. - have some infrastructure set up for reporting errors to the user (note that this will not be complete. Error handling can sometimes hide errors/issues from you, so you want to wait to have this complete)
- allow the user to input an expression from a file and display the image generated from that expression.
- handle order of operations for two binary operators with different levels of precedence (e.g., addition and multiplication); set up the code so that it is easier to update as you add new operators. There will be a bunch for the final implementation. Set up the code for easier additions now, considering design principles and the steps required for adding new operators (which will be more than just binary operators).
It is implied, but I will make it explicit: each member of the team should be contributing toward this deliverable, in code and in other ways.
All test cases should compile. Again, if you are using test-driven development, put those tests in a separate, marked file or comment those tests out.
The most difficult parts of this deliverable are assignments and the image-manipulation function. However, you get a much-clearer understanding of the code.
Create
a new release of the application. Tag it as version v0.6
and
title it as Intermediate Functionality
. The target should
be the main
branch. Now, you'll be able to easily refer
back to this version.
Check out this version of the code to confirm that it works--that there aren't missing files that are only on your computer.
The team will demo this application in class to the professor and discuss
- what they have completed since the preliminary implementation deadline
- show their favorite expressions (that have been saved into files in the expressions/ directory of the git repository)
- how their design has changed
- their extensions plan (if it has changed or if it is the same) and how they plan to approach the extensions
Final functionality (45%) - Team
Due: Determined by team, no later than Dec 13 (Thursday of exam week) at 11:59 p.m.
This should be a complete working version of the Picasso
specification and your chosen extensions. Since there is no demo part
of this deliverable, include documentation of extensions and how to use
them in
the README.md
file.
Create
a new release of the application. Tag it as
version v1.0
and title it as Final
Application
. The target should be the main
branch. Now, you'll be able to easily refer back to this version.
Check out this version of the code to confirm that it works--that there aren't missing files that are only on your computer.
Collaboration and Contribution. 20% of the grade you receive for the final functionality is reserved for your individual contributions to the team/the project. Everyone should contribute significantly to the project. This should be reflected somewhat in the contributions to the GitHub repository, although that is a flawed metric. Your final analyses will paint a broader picture of your contributions--from yours and your teammates' perspectives. Furthermore, could your team rely on you? Could you collaborate on design decisions? Did you make everyone on the team feel valued?
Ungraded Objectives. Everyone has at least one part where they can say "I made this!". Everyone understands all of the code and its design--well, at least 90% and at a high level, which should be clear from the post-mortem analysis.
Post-mortem (15%) - Individual
Due: Friday, December 13, noon (end of exams).