Category Archives: pixel manipulation

QuickShader Micro-Lib

In 2015 I created QuickShader… which just takes the boilerplate out of showing a shader in the browser. Here are a few examples:

Also posted in Graphics, Math, color, glsl, graphics algorithms, html5, javascript, motion | Tagged , | Leave a comment

SVG to Canvas (good trick)

Awhile back, I wrote some collision detection code that blitted existing interactive SVG to Canvas and then used the pixel data to figure out various aspects of the relationships between arbitrary SVG nodeTypes. A really simple trick I used can be seen in this pen:

The trick is to load the svg data into an image as a datauri. There are other tricks like this - one of which is using an svg `foreignObject` to blit html to canvas:

https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Drawing_DOM_objects_into_a_canvas

There were some browser issues at the time with this. The main one being IE 10/11 didn’t really work (tainted canvas if I recall correctly). The `foreignObject` trick didn’t work with image xlink:hrefs in safari at the time… (weirdly if you opened the dev tools it would start to work) anyway…

I ended up forking canvg for various cases. canvg is really cool… just a note, a coworker of mine went in at some point and optimized it like crazy and improved the perf a good deal by “drying things up”. Maybe I’ll suggest that he submit his optimizations at some point.

Also posted in Graphics, html5, javascript, misc, svg | Tagged , , , , , , | Leave a comment

Frame Differencing

Actionscript:
  1. [SWF(width = 800, height= 600)]
  2. var sw:Number = 800;
  3. var sh:Number = 600;
  4. var pixelNum:int = sw * sh;
  5. var blurAmount:Number = 10;
  6. var pnt:Point = new Point(0,0);
  7. var rect:Rectangle = new Rectangle(0,0,sw,sh);
  8.  
  9. var canvas:BitmapData = new BitmapData(sw, sh, false, 0x000000);
  10. var buffer:BitmapData = new BitmapData(sw, sh, false, 0x000000);
  11. var feed  :BitmapData = new BitmapData(sw, sh, false, 0x000000);
  12. var prev  :BitmapData = new BitmapData(sw, sh, false, 0x000000);
  13.  
  14. var frame:Bitmap = new Bitmap(canvas, "auto", true);
  15. addChild(frame);
  16.  
  17. var cam:Camera =  Camera.getCamera();
  18. cam.setMode(sw,sh,12);
  19. var video:Video = new Video(sw, sh);
  20. video.attachCamera(cam);
  21.  
  22. cam.addEventListener(ActivityEvent.ACTIVITY, onActivityStart);
  23.  
  24. function onActivityStart(evt:ActivityEvent):void {
  25.     addEventListener(Event.ENTER_FRAME, onRun);
  26.     cam.removeEventListener(ActivityEvent.ACTIVITY, onActivityStart);
  27. }
  28.  
  29. function onRun(evt:Event):void{
  30.     buffer.draw(video);
  31.     feed.copyPixels(buffer, rect, pnt);
  32.     buffer.draw(prev, null, null, BlendMode.DIFFERENCE);
  33.     prev.draw(video);
  34.     canvas.copyPixels(buffer, rect, pnt);
  35. }

This snippet shows a simple method to do frame differencing with a web cam. This is useful for detecting which areas of the screen have change from frame to frame. This post assumes you pretty much know what frame differencing is.


You can view an swf file here:

This is one of those things I do a good deal but never wrapped up into a library... it's easy (for me at least) to forget exactly how to set it up from scratch.

I added an extra buffer because it's pretty common that you'll want to other things aside from just frame differencing alone, so it's nice to have it all wrapped up in a buffer BitmapData that you can use elsewhere. If speed is a real concern, you can do away with the buffer and just use the canvas instead - depending on what kind of analysis your doing on the frame differenced image it may be trickier without the buffer.

Also posted in BitmapData, Video, graphics algorithms | Tagged , , | 5 Comments

~20,000 Rollovers

Actionscript:
  1. [SWF(width = 500, height = 500, frameRate = 30)]
  2.  
  3. var canvas:BitmapData = new BitmapData(stage.stageWidth,stage.stageHeight,false, 0xFFFFFF);
  4.  
  5. var indexCanvas:BitmapData = new BitmapData(stage.stage.stageWidth, stage.stageHeight, false,
  6.                                             0xFFFFFF);
  7. addChild(new Bitmap(canvas));
  8.  
  9. var s:Shape = new Shape();
  10.  
  11. var lineData:Array = [];
  12. var dataIndex:int = 0;
  13.  
  14. trace(0xFFFFFF - 1)
  15. var totalLines:int = 20000;
  16. var iterations:int = 9;
  17. var linesPerIter:int = totalLines / iterations;
  18.  
  19. var xp:int = stage.stageWidth / 2;
  20. var yp:int = stage.stageHeight / 2;
  21.  
  22. var stepAmt:Number = 60;
  23. var halfStepAmt:Number = stepAmt / 2;
  24.  
  25. addEventListener(Event.ENTER_FRAME, onDraw);
  26. function onDraw(evt:Event):void {
  27.      if (lineData.length <totalLines){
  28.         generateData(linesPerIter);
  29.      }else{
  30.         stage.quality = "high";
  31.         addChild(s);
  32.         s.x = 0;
  33.         s.y = 0;
  34.          
  35.         removeEventListener(Event.ENTER_FRAME, onDraw);
  36.         addEventListener(Event.ENTER_FRAME, onRun);
  37.      }
  38. }
  39.  
  40. function onRun(evt:Event):void {
  41.    var currentIndex:int = indexCanvas.getPixel(mouseX, mouseY);
  42.    var currentLine:Array = lineData[currentIndex];
  43.    
  44.    s.graphics.clear();
  45.    if (currentIndex != 0xFFFFFF){
  46.           s.graphics.lineStyle(3, 0xFF0000);
  47.           s.graphics.moveTo(currentLine[0], currentLine[1]);
  48.           s.graphics.lineTo(currentLine[2], currentLine[3]);  
  49.    }
  50. }
  51.  
  52. function generateData(num:int):void{
  53.     var rxA:int, rxB:int, ryA:int, ryB:int;
  54.     var g:Graphics = s.graphics;
  55.     for (var i:int = 0; i<num; i++){
  56.         rxA = xp;
  57.         ryA = yp;
  58.        
  59.         xp += Math.round(Math.random() * stepAmt) - halfStepAmt;
  60.         yp += Math.round(Math.random() * stepAmt) - halfStepAmt;
  61.        
  62.         if (xp> stage.stageWidth){
  63.             xp = stage.stageWidth - halfStepAmt;
  64.         }else
  65.         if (xp <0){
  66.             xp = halfStepAmt;
  67.         }
  68.         if (yp> stage.stageHeight){
  69.             yp = stage.stageHeight - halfStepAmt;
  70.         }else
  71.         if (yp <0){
  72.             yp = halfStepAmt;
  73.         }
  74.        
  75.         rxB = xp;
  76.         ryB = yp;
  77.          
  78.         lineData[dataIndex] = [rxA, ryA, rxB, ryB];            
  79.         s.x = rxA;
  80.         s.y = ryA;
  81.         var endX:Number = rxB - rxA;
  82.         var endY:Number = ryB - ryA;
  83.         var m:Matrix = s.transform.matrix;
  84.         g.clear();
  85.         g.lineStyle(1, 0x000000, 0.3);
  86.  
  87.         g.lineTo(endX, endY);
  88.         stage.quality = "high";
  89.         canvas.draw(s, m);
  90.        
  91.         g.clear();
  92.         g.lineStyle(3, dataIndex);
  93.        
  94.         g.lineTo(endX, endY);
  95.         stage.quality = "low";
  96.         indexCanvas.draw(s, m);
  97.        
  98.         dataIndex++
  99.     }
  100. }

I'm working on a data visualization that contains a long path made up of approximately one million points. There is some information associated with every two sets of coordinates that needs to be displayed when the user rolls their mouse over any part of the line.

I took a little time to think about the best way to do this and came up with a few techniques. The first one I tried seems to work nicely - this snippet is the proof of concept for that first technique. I tested this snippet with 1,000,000 xy coordinates and it works nicely. It takes a little while to draw though, so for the purposes of this demo I've just included 20,000 coordinates.

Have a look at the swf over at wonderfl.net

The way this works is by drawing lines to two different BitmapData instances. I draw anti-aliased slightly transparent lines to a BitmapData instance called "canvas" (this is added to the display list) - I then draw aliased lines to a BitmapData called "indexCanvas" (this is never added to the display list) - each aliased line uses an incremental value for its color - this incremental value is also the index for a two dimensional array containing the coordinate information for the aliased line. I use getPixel() on the "indexCanvas" and use the return value as the index for the 2D array. The data from the 2D array is used to draw a red line with the graphics class. This technique enables you to have many many rollovers and all you ever have to do is call getPixel() and use the returned color value to look up info about what you're mouse is touching.

There are a few cool ways this could be repurposed and this is really only one solution to the problem of having many many things that you need to be able to rollover... there are others that don't use BitmapData at all... I may write those up in the next couple of days.

Also posted in BitmapData, Data Structures, UI, arrays, display list, graphics algorithms, matrix, misc, return values | Tagged , , | 2 Comments