The Code
We started off with the code to parse picture data down into something interpretable by the Fanuc system. The major limitation here was the memory constraints of the industrial arm. There was no direct connection between the Fanuc and PC we intended to use to process the image, we were limited to the 500kb internal memory of the Fanuc arm control system (potentially there exists a workaround wherein you can automatically increment the position registers in the Fanuc teach pendant using a National Instrument sensory control board, but we decided doing it the quick and dirty way would be perfectly sufficient).These memory limitations ultimately caused us to decide to have the robot draw crosshatching, rather than the outline of the processed pictured, as we concluded this technique would required fewer path points. The image processing was all written in my go-to language Java. I whipped up a simple application that would load in an image, convert the image to greyscale, let you chose the color values for each threshold level, then threshold the image. Rather than use a good machine vision library like openCV, I decided to make my own, mainly for shits and giggles. I implemented blob detection using a simple floodfill algorithm, in which pixels within a specified color threshold are marked. Each one of these marked pixels that hasn't been handled becomes a blob, and then searches the space directly around it for other pixels within the color threshold, adding them to its blob and marking them as processed (so they don't become their own blobs). At the same time, the outline of blobs are detected whenever no more similarly colored blobs are found in the pixel's neighbors.
With this information, it was then easy to generate a number of lines and find their intersects with the blobs of each threshold level. These intersections were marked as Java 2D points and passed on to a matrix coordinate transformer and marshaling program. The matrix coordinate transformer rotated, scaled, and translated the data points to a zone within the workspace of the Fanuc arm. The marshaling program automatically added in safety points between the line segments in such a way that the pen was lifted from the paper during the transition between the lines. It then wrote the points to a .ls file in the correct syntax. These files were then compiled with Fanuc's proprietary software, ROBOGUIDE, which produced .tp files which were uploaded to the Fanuc's teach pendant and executed.
The Gripper
The gripped was designed to allow for compliance along the longitudinal axis of the pen. This way, we'd be able to keep the pen in constant pressure against the paper.The Results
The system worked quite well, drawing images to satisfactory detail.
The only issue was execution time. We found that increasing the jog speed beyond a certain speed caused the produced lines to be splotchy. Ultimately, it took about two hours to produced an image of the complexity shown in the video below
Aaaaannnnndddd, the final result!
finally, some technical documents and source code!
TechnicalPaper
gripper report
AnimatronicArtistSource
GenericGridLibrary
