Sunday, January 4, 2015 goes secure

I've had a self signed cert on which had some limited use.  There should be a real SSL cert now.

EDIT: the map issue has been fixed

Thursday, January 1, 2015

Scaling up panotools some more

To recap, some things that I currently do to help stitch large IC photos with pr0ntools:
  • creates baseline .pto.  Instead of doing n to n image feature match, only matches features to adjacent images.  This reduces feature finding from O(n**2) to O(n)
  • creates image tiles from .pto file. This allows me to stitch large panoramas with limited RAM
  • Some misc wrapper scripts such as that helped work around some peculiarities of tools like PToptimizer
For large panoramas, these were still a few problematic parts:
  • Big problem: PToptimizer is O(n**2).  Very slow for large image sets.  IIRC 1200 images took a week to optimize
  • Cropping and rotating .pto in Hugin is unnecessarily slow since I only use the outer images for alignment.  With n images only about O(n**0.5) are used
  • was single threaded.  I recently picked up a beefy machine at a flea market and now had an incentive to take advantage of the extra cores
 Other issues:
  • I couldn't get cpfind to produce good results so I used closed source autopano at its core to do feature matching.  This has a lot of hacks to work through wine which is complicated and hard to setup
Lets see what we can do about these.

Regarding optimization, panotools has a tool called autoptimizer with a nifty feature: "-p Pairwise optimisation of yaw, pitch and roll, starting from first image."  This sounds a lot better than the O(n**2) optimization that PToptimizer dos.  Unfortunately, as stated, and, verified through my trials, it does not work for position.

So, I decided to try to see what I could do myself.  Most of the time is spent getting the panorama near its final position.  However,since the images are in an xy grid, we have a pretty good idea of where they'll be.  But better than that, since the images are simple xy transpositions we should be able to estimate pretty close to the final position simply by lining up control points from one image to another.  By taking the average control point distance from one image to another, I was able to very quickly construct a reasonably accurate optimized estimate.  The upper left image is taken as coordinate 0, 0 and everything is relative to it.  I then iterate the algorithm to fill in all remaining positions.  Presumably I could iterate the algorithm additional steps to optimize it further but at this time I simply hand off the pre-optimized project to PToptimizer.

This has a dramatic performance impact.  Here's some results from a project with 805 images:
  •  pr0npto --optimize: real    488m47.457s
    • Normal PToptimizer
  • pr0npto --pre-opt: real    0m58.843s
    • Pre-optimizer only with no PToptimizer final pass
    • rms error: 0.293938367201598 units
  • pr0npto --pre-opt-pt: real    29m39.604s
    • Pre-optimizer followed by PToptimize
    • rms error:  0.215644922039943 units
    • Took 3 iterations
--optimize and --pre-opt-pt should produce about the same final result.  I forgot to check what the final optimization score of --optimize was.  Anyway, above results show that the pre-optimizer was able to pre-position to better than average 1/3 pixel.  A final optimizer pass was able to slightly improve the result.

The next problem was crop/rotation.  There are two problems:
  • Its unnecessarily slow to create thumbnails for many images, most of which will just be thrown away
  • I'm not sure what the order of the preview algorithm is, but its much worse than O(n) on my laptop:
    • 74 images: 10.6 img / sec
    • 368 images: 2.9 img / sec
My solution: create a wrapper script (pr0nhugin) that eliminates the unused images.  This won't fix hugin's O(>n) issue but mitigates it by keeping n low.  Basically it deletes the unused images and opens a sub-project with the reduced image set.  After hugin closes the new crop/rotate parameters are merged back into the original project.

Sample results:
  • Original project: 368 images in 128 seconds
  • Reduced project: 74 images in 7 seconds
Original project in fast pano preview:

Reduced project in fast pano preview:

128 seconds isn't so bad but this is a smaller project and gets pretty bad for larger projects.  Could also probably eliminate every other image or something like that to speed it up more if needed.

Finally, I figured out what I was doing wrong with cpfind.  It was quite simple really, I basically just needed to use cpclean to get rid of the control point false positives.  Comparison of optimization results:
  • pr0nstitch w/ autopano: rms 1.09086625512561 units
  • cpfind/cpclean optimize pitch/raw/yaw xyz: rms 6.25849993203006 units
    • Only xyz should change, its optimizing things it shouldn't
  • cpfind/cpclean optimize xy: 0.510798407629321 units
    • IIRC this was through new pr0nstitch but can't recall for sure
Some quick tests  show that cpfind/cpclean meets or exceeds autopano performance.  Given the pains of coordinating through WINE (they have a Linux version but it doesn't work as well as the Windows one).

Now with the backend tool working smoother I was able to parallelize feature finding easier.  Basically pr0nstitch now spawns a bunch of worker threads that calculates features between two images.  A main thread merges these into a main project as workers complete matching sub-images.

There is also a stitch wrapper script that pulls these tools into a recommended workflow.

Summary of new round of improvement:
    • Parallelized
    •  Switch to panotools feature finding/cleaning.  Now fully open source and much easier to setup
  • new efficient --pre-opt-pt optimizer
  • hugin wrapper to edit reduced .pto files for fast editing
  • Added "stitch" workflow script

    Friday, December 12, 2014

    Need help: data

    The bulk data store has gone down:

    Do you have any data from this before it went down?  If so, please contact me: JohnDMcMaster at gmail dot com

    Wednesday, December 10, 2014

    They're multiplying!

    I've been continuing to troubleshoot the Super IIIA SEM and, by using putty, have discovered that the leak is in the column.  I'm having some trouble figuring out how to safely disassemble it to fix the leak.  More on that later.

    In the meantime, I've been exploring alternate paths.  My ISI Super IIIA had a baby SEM!

    I found someone selling an ISI M-7, a small tabletop SEM from the mid 70's.  The catch?  I had to drive from the SF Bay Area down to LA to pick it up (385 miles each way).  He is a semi-retired Cambridge SEM technician, not sure how the ISI unit got into the mix.  The landscape:

    Its the blue unit on the right.

    I was already aware that the PMT assembly had sustained some damage but was unsure how substantial.  Unfortunately the PMT tube itself was cracked:

    Fortunately I found one reasonably cheap on eBay that's in the mail.  IIRC the Super IIIA uses the same PMT so I could borrow it from that if necessary.  Additionally, an IC came lose in the PMT base that doesn't really match with the socket.  Need to do some work to figure out what happened.

    After sitting in the garage for some years it was a bit of a mess.  For example, here's a close up of the vacuum manifold:

    After some cleaning it looked a lot better but still has a lot of rust:

    Next, I wired everything up to the best I could given that not all the cables were still labeled:

    I love the enormous military grade connectors.  It didn't come with an AC cord (requiring a military grade connector) but I was able to borrow one from the S3A.  A few other things are required in addition to the core equipment shown above:
    • Roughing pump
    • Diffusion pump circulation pump / cooling
    • Variac to step down 120VAC to 100VAC input voltage

    Turned the power on and no smoke came out.  The "replace anode" light came on which I'm not sure what to make of yet.

    Unfortunately I don't have a 5/8" adapter for my vacuum pump so I can't attempt to rough it down.  Ordered a $10 KF16 adpater from Tedpella, waiting for it in the mail along with the PMT replacement.

    Sunday, October 5, 2014

    Updated AmScope MU800 driver

    Can be found here:

    Hopefully will get merged into the kernel soon.  Over previous candidate this fixes some minor formatting issues and removes a serial number comparison check (read: only would have worked with my camera).

    I also have traces from an MU300 and might merge those in soon.  Shout out if you'd really love to see MU300 supported on Linux. gets a facelift!

    Thanks to Enrique Murias Fernández!, the original "2 minutes in vi" homepage has finally been replaced with something more proper:

    Getting a real SSL cert is on my radar.  I tried to get one from StartSSL but they never responded to me  Unfortunately, it may cost more than the web hosting...

    Wednesday, October 1, 2014

    Confocal microscope bringup

    One of the biggest issue with the process of digitizing ICs is that, at least with the tools/people available to me, polygon capture is mostly done by hand.  If you look at most of the automated polygon capture systems they either rely on SEMs (sharp images) or confocal imaging (planar view reduces background noise).  SEM imaging has its place and eventually I'll having a working SEM (sniff...that's for another post)

    Most confocal microscopes out there are intended for biological imaging.  You can, for example, tag cells with fluorescent dies and build up a 3D image.  In some cases you can even image live cells and get some pretty sweet 3D videos.

    Of metallurgical / epi illuminated by far the most popular model out there is the Technical Instruments K2 IND.  I tried to get some from local industrial auctions but kept getting outbid.  After some online searching/negotiating I was able to get one at a reasonable price off of eBay:

    The system didn't come with any objectives and is not computer controlled but is otherwise complete (also includes power supply, not pictured).  I saw other scopes using Nikon BD Plan objections but I'm unclear if this is wasteful over the Nikon M objectives: I don't see a way to use darkfiled functionality.  Anyway, I picked up a Nikon BD Plan 20x objective and the system did in fact work.

    As an impulse buy, I had previously purchased a heavy duty Technical Instruments XY stage at industrial auction:

     The linear stage is actuated by Compumotor M-57 stepper motors and is equipped with quadrature equivalent output RSF Elektronik glass linear encoders (10 um resolution):

    The encoders are nice but I can generally get away with running stepper motors in open loop so not sure if I will use them.  That said, linear encoders can be used to eliminate backlash which would be nice.

    It also came with matched Compumotor M-57 motor controllers:

    They run open loop and therefore do not take into account either encoder. A complete system with heavy duty stages, cables, drivers, etc and I think I only paid something like $120 net for it.  Also came with random nosepiece which got thrown in my junk drawer.

    Finally, not shown is the Z column.  Its not motorized but its pretty heavy duty.  Its unclear to me if this setup was intended to be used with the K2 IND, but it looks heavy duty enough to support one.  It seemed a good match for this microscope and I began the conversion process by (gasp) cutting the Nikon microscope body in half so I could re-mount it to my Technical Instruments stand.  Target:

     But how to clamp this?  I have a smallish table top mill (Sherline 2000) with vices nowhere near the size of the stand.  But it just needs to be stable without super precision which opens things up a bit.  What if I could just hold it down with some plumbers tape?

    So I took a small unused optical table and wrote a CNC drill program to drill some countersunk holes so that I could mount it to the table:

    I can't remember how I drilled the first few holes to bootstrap it...

    Anyway, this allowed me to securely fasten the remaining microscope body and clean up the sawzall cuts:

    Got busy at work, broke my CNC mill with some unrelated work (its fixed now) and time passed.  I was looking into getting objectives and saw another K2 IND at industrial auction.  It turned out that it was likely going to cost me less to get another K2 IND with objectives at industrial auction than to buy objectives off of eBay.  And I ended up winning this system:

    They're multiplying!

    But this new system, a Zygo KMS 310 RT, is actually quite nice.  Intended for mask inspection, seems to have been some partnership between Zygo and Technical Instruments.  Anyway, it came with a lot of stuff:
    • Infinity corrected K2 IND (KMS310)
    • 4 Nikon CF Plan objectives (very nice)
    • XY DC servo motors with glass linear encoders
    • Linear and quadrature encoded DC servo Z motor (cause the thing weights so much its not manually actuated)
    • Piezo Z stage for fine focusing
    • Vibration isolated table w/ CRT.  This may be why it went for a bit low price: people maybe didn't want to deal with getting rid of this
    • No control system (probably drove down price)
    • Came with cables, some of which were cut
    While I couldn't find much information on the KMS 310 RT, I did find some pictures of Zygo KMS450i which looks to have identical linear stages.  Its a monstrosity of a control box and probably just as well that I replace it with just what I need

    But what do I need?  A bit of a problem: all of the XY stages I've ever worked on have been open loop stepper systems.  I'm now confronted with DC servos, a whole different paradigm.

    My old boss liked to say that cheating is a valid engineering strategy.  Can we cheat?  yes we can!  Here are the two Technical Instruments bases I have:

    They are made by the same company and the footprints look awfully similar.  Wonder if I can take the stepper stage and transplant it onto the KMS 310 RT?  Yep!

    Now just the Z axis to deal with.  For starters I just attached a benchtop power supply and was able to get it to move around (its a DC motor after all).  But not very convenient.  If I was really determined I may have been able to remount a stepper in its place.  But its a good excuse to learn more about servo systems.

    After doing some research and talking to people, Geckodrive Motor Controls seems to be pretty good quality and reasonable cost.  While most of their business is in stepper drivers, it turns out that they make a DC servo drive, the G320X Digital Servo Drive:

    I ordered two of them, intending one to be used for X and one for Y.  But with the XY problem solved for the time being it seemed fitting to use it on the Z axis.

    Next challenge: figure out motor pinout.  After some multimeter action I was able to trace the wires on the motor to the umbilical cord coming off of the KMS 310 RT (which I cut in half since I didn't want to buy $ circular connectors).  The encoder pinout was printed on the encoder but I had some problems getting it to work.  After some looking around, I found the website for the motor and learned the encoder was open collector.  After adding some 1k pullup resistors the encoder signals looked good on my scope and the Gecko accepted them.  Final test setup:

    Last piece of the control system: need to generate step/direction pulses to the compumotor XY drivers as well as the gecko Z driver.  I already have a low cost indexer (pr0ndexer) I use on my Olympus BH2, so I simply made another one of those:

    Fired up my cnc microscope program...and it worked!  All the gains are set for my BH2 system but those should be reasonably easy to adjust.

    Proof of concept system overview:

    But I'm not in clear water yet.  The original Nikon K2 IND seems to work fine in all accounts.  The KMS 310 RT works in normal epi mode, but doesn't let enough light through to be usable in confocal mode.  I could swear it worked before, not sure.  The problem should be solvable one way or the other with the worst case being that I replace it with the Nikon K2 IND.

    Other future plans: I'd still like to get the original DC servo linear stages working.  They use sine-cosine encoders which are a bit more complex to use.   I also need to incorporate a levelling element (probably a mirror mount) so that chip remains in focus across the scan.  Once that's in place I can do my first scan.

    Until next time!