Friday, March 04, 2011

Silly user, tricks are for programmers!

At the one and (thankfully) only systems department meeting I attended at the library over a month ago there was a short conversation about how systems installed an obscure version of perl (not ActiveState or Cygwin) on a cataloging department Windows XP box, so a cataloger could run a magic perl script handed down from the ages. The magic perl script processed a file of MARC records from our "books on order" vendor to update each record with the correct Auburn location code for where the record's book would be shelved.

I volunteered at the time that Jack or I could whip out a java or C# program to replace the perl script. I was warned that this was complex code that tied together purchase orders, bib, holdings, and item records. I had to see this amazing piece of work!

When I finally got around to looking at the perl script and a sample input I found about 50 lines of perl and a simple file full of BIB records. I knew it!

Anyway, I wrote a little webstart app (MarcConvert - more info here) that could probably replace the perl script. MarcConvert reads an input file of records in MARC or MARCXML format, and writes the records back out to an output file in MARC, MARCXML, or SIMPLE (readable) format while optionally applying a "location filter". I'll e-mail a few people at the library to see if they want to give it a try. MarcConvert should be useful in a few ways.

  • First, MarcConvert can transform a file of MARC records to a readable format, so a cataloger can take a quick look at what is actually in the file.
  • Next, if a cataloger wants to make a simple change to a file of MARC records, then he/she can just convert the file to XML, make the changes to the XML in a standard editor, then convert the XML file back to MARC.
  • Finally, a cataloger can apply the "location filter" to a MARC file without installing perl.

Thursday, March 03, 2011

jre6 update 24 sucks

It's always annoying when something outside my control changes, and breaks my code. I received an e-mail the other day from my former coworker at the library reporting that one of the little tools I wrote stopped working on some machines. I suggested a few things to try, but I assumed that something ridiculous happened at the library, and they would fix it up. By the end of the day they let me know that web start with the new java 6 update 24 threw a SecurityException when launching my tool.

I at first could not reproduce the problem running a copy of the library tools, but that was just because I was running a pre-update version of java. My Windows 7 laptop has 3 java installs - a 64 bit jre, a 64 bit jdk, and a 32 bit jre - only the 32 bit jre has update 24. My netbeans developer environment and powershell command line both point at the pre-update jdk, but Chrome's java plugin runs the 32 bit jre with update 24. I was already annoyed at this point. Ugh.

Google helped me narrow the problem down to web-start's interaction with the Apache felix OSGi implementation that I use. I wrote up a little test case (below), and submited a ticket to Oracle's java team.

package littleware.demo;

import java.io.PrintWriter;
import java.io.StringWriter;
import java.net.URL;
import java.net.URLClassLoader;
import java.sql.Connection;
import java.util.HashMap;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.swing.JFrame;
import javax.swing.JOptionPane;
import javax.swing.JScrollPane;
import javax.swing.JTextArea;
import javax.swing.SwingUtilities;
import javax.swing.WindowConstants;
import org.apache.felix.framework.Felix;


public class JavaToy {
    private static final Logger log = Logger.getLogger( JavaToy.class.getName() );

    public static class AppRunner implements Runnable {
        public void run() {
            final StringWriter swriter = new StringWriter();
            final PrintWriter pwriter = new PrintWriter( swriter );
            pwriter.append( "Class path: " ).append(
                    System.getProperty( "java.class.path" )
                    ).append( "\n\n-------------------------\n" );
            final ClassLoader classLoader = this.getClass().getClassLoader(); //Thread.currentThread().getContextClassLoader();
            if ( classLoader instanceof URLClassLoader ) {
                pwriter.append( "URLClassLoader:\n" );
                for ( URL url : ((URLClassLoader) classLoader).getURLs() ) {
                    pwriter.append( url.toString() ).append( "\n" );
                }
                pwriter.append( "\n--------------------------------\n" );
            }
            try {
                Class.forName( "bogus.DoesNotExist" );
                pwriter.append( "No exception thrown on bogus class load\n" );
            } catch ( Exception ex ) {
                pwriter.append( "Caught exception loading bogus class: " ).append( ex.toString() ).append( "\n" );
                ex.printStackTrace(pwriter);
            }
            pwriter.flush();
            final JFrame jframe = new JFrame( "Webstart test" );
            final JTextArea jtext = new JTextArea( swriter.toString(), 20, 40 );
            jframe.add( new JScrollPane( jtext ) );
            jframe.pack();
            jframe.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
            jframe.setVisible(true);
        }
    }
    public static void main( String[] args ) {
        try {
            log.log( Level.INFO, "Launching felix!" );
            (new Felix(new HashMap<String, Object>())).start();
            Thread.sleep( 2000 );
        } catch (Exception ex) {
            log.log(Level.SEVERE, "Caught exception", ex);
            System.exit(0);
        }
        SwingUtilities.invokeLater( new AppRunner() );
    }
}

Web-start with a pre-update-24 jre throws the expected ClassNotFoundException and yields this result:

Class path: C:\Program Files\Java\jre6\lib\deploy.jar

-------------------------
URLClassLoader:
file:/C:/Users/pasquini/Documents/Code/JavaToy/dist/JavaToy.jar
file:/C:/Users/pasquini/Documents/Code/JavaToy/dist/lib/felix-2.0.4.jar
http://felix.extensions:9/

--------------------------------
Caught exception loading bogus class: java.lang.ClassNotFoundException: bogus.DoesNotExist
java.lang.ClassNotFoundException: bogus.DoesNotExist
...

An update-24 jre throws a troublesome SecurityException with this result:

Class path: C:\\Program Files (x86)\\Java\\jre6\\lib\\deploy.jar

-------------------------
URLClassLoader:
file:/C:/Users/pasquini/Documents/Code/JavaToy/dist/JavaToy.jar
file:/C:/Users/pasquini/Documents/Code/JavaToy/dist/lib/felix-2.0.4.jar
http://felix.extensions:9/

--------------------------------
Caught exception loading bogus class: java.lang.SecurityException: Permission denied: http://felix.extensions:9/bogus/DoesNotExist.class
java.lang.SecurityException: Permission denied: http://felix.extensions:9/bogus/DoesNotExist.class
...

The bug robot sent me a nice response for my trouble.

Dear Java Developer,

Thank you for your interest in improving the quality of Java Technology.

Your report has been assigned an internal review ID of 1990127, which is NOT visible on the Sun Developer Network (SDN).

Please be aware that the large volume of reports we receive sometimes prevents us from responding individually to each message.

If the information is determined to be a new Bug or RFE, or a duplicate of a known Bug or RFE, you will receive a followup email containing a seven digit bug number. You may search for, view, or vote for this bug in the Bug Database at http://bugs.sun.com/.

If you just reported an issue that could have a major impact on your project and require a timely response, please consider purchasing one of the support offerings described at http://developers.sun.com/services/.

Bla bla bla ...

Fortunately, the felix user mailing list had a thread on the issue, and some cool Karl Pauls guy pointed me in the right direction to throw together a little patch (see below) that got me back in business.

Hurray for open source! Of course now its 3 days of head banging, trying random hacks, and considering dropping felix or web-start later. I'm too relieved to be bitter! Oracle actually closed a related ticket as "Not a bug", because the http://... URL in the ClassLoader path violates the web-start single-origin policy - now more strictly enforced in update 24. That looks like some lazy Oracle developer dodging work to me, because a signed web-start app granted all-permissions can do almost anything (open network ports, erase the disk, write and launch a program, whatever), so what's the deal with the SecurityException scanning the class path ?

Re: Java 6 Update 24 breaks Felix
Reuben Pasquini
Thu, 03 Mar 2011 10:00:46 -0800

Here's an ExtensionManager patch that got me back in business with webstart

(don't know what's up with the sun.org.moz... import in Felix.java):

---------------------

C:\Users\pasquini\Documents\Code\felix-framework
> svn diff -x -w
Index: src/main/java/org/apache/felix/framework/Felix.java
===================================================================
--- src/main/java/org/apache/felix/framework/Felix.java (revision 1076687)
+++ src/main/java/org/apache/felix/framework/Felix.java (working copy)
@@ -75,8 +75,8 @@
 import org.osgi.framework.hooks.service.ListenerHook;
 import org.osgi.service.packageadmin.ExportedPackage;
 import org.osgi.service.startlevel.StartLevel;
-import sun.org.mozilla.javascript.internal.UintMap;

+
 public class Felix extends BundleImpl implements Framework
 {
     // The secure action used to do privileged calls
Index: src/main/java/org/apache/felix/framework/ExtensionManager.java
===================================================================
--- src/main/java/org/apache/felix/framework/ExtensionManager.java
(revision 1076687)
+++ src/main/java/org/apache/felix/framework/ExtensionManager.java
(working copy)
@@ -87,6 +87,10 @@

     static
     {
+        // Allow system-property based disable of ExtensionManager.
+        // Adding http://felix.extension:9 to the class path causes
problems
+        // in webstart which enforces a "single origin" policy as of jre 6
update 24.
+        if( System.getProperty( "felix.extensions.enabled", "true"
).equalsIgnoreCase( "true" ) ) {
         // pre-init the url sub-system as otherwise we don't work on
gnu/classpath
         ExtensionManager extensionManager = new ExtensionManager();
         try
@@ -113,7 +117,10 @@
             extensionManager = null;
         }
         m_extensionManager = extensionManager;
+        } else {
+            m_extensionManager = null;
     }
+    }

     private final Logger m_logger;
     private final Map m_headerMap = new StringMap(false);

---------------------------



Then add

           <property name="felix.extensions.enabled" value="false" />
in the .jnlp <resources> block, and I'm back in business.

Thanks for pointing me in the right direction.

Hope something like this makes it into the  next release ... ?



Cheers,

Reuben


On Thu, Mar 3, 2011 at 10:52 AM, Reuben Pasquini <reu...@frickjack.com>wrote:

> I agree it sounds like an Oracle bug, but a workaround would be great.
>
> A config parameter that disables extension bundles works for me too.
>
> I embed felix in a very limitted way to manage the startup/shutdown
> lifecycle
>
> via BundleActivators, so I don't think my apps will miss the extension
> bundles.
>
> Let me know when you have a patch checked in, and I'll give it a try ...
>
> or if you think it's a small change, then point me at the right part of the
> felix
>
> repo, and I'll try to throw together a patch for you.
>
> Thanks for the help!
>
>
>
> Cheers,
>
> Reuben
>
> On Thu, Mar 3, 2011 at 3:16 AM, Karl Pauls <karlpa...@gmail.com> wrote:
>
>> On Thu, Mar 3, 2011 at 10:08 AM, Bauersachs Ingo
>> <ingo.bauersa...@fhnw.ch> wrote:
>> >> Problem is this doesn't make sense to me as the app has allpermission
>> >> (or sets the security manager to null even). Why enforce this then? It
>> >> sounds like a bug (and there are several at oracle by now). Lets see
>> >> what they do
>> >
>> > Indeed, this makes no sense.
>> >
>> >> I'll try to find a workaround on the weekend and if
>> >> not, I'll probably add a property to disable extension bundles (which
>> >> would make the url go away - only extension bundles will not work).
>> >
>> > 6u24 is out and I needed a workaround fast. Changing the extension url
>> from http to file seems to satisfy the same origin policy. If my change
>> doesn't cause other side-effects it might even be a permanent solution. (I
>> have far to less experience with OSGi/Felix to really comment on this)
>>
>> If it works for you it is ok for now - it is not the permanent
>> solution as there will be other issues (there is a reason we do what
>> we do there - related to extension bundles). But if i can't find
>> something better the solution will be close (i.e., provide a way to
>> disable extension bundles -> don't register this url at all).
>>
>> regards,
>>
>> Karl
>>
>>
>> > Regards,
>> > Ingo

Monday, February 28, 2011

Tweaking Google Code's Issues Gadget

I'm a big fan of Google Code where I host the littleware project. I've been playing around with gcode's issue tracker trying to figure out some way to embed the tasks associated with a code sprint into the sprint's overview documentation, and I eventually stumbled upon a nice issue-tracker gadget in a Gadgets for Google Code bLog entry that almost does what I want, but not quite!

Fortunately, it turns out that the issue-tracker gadget is one of the gadgets developed by google's code-hosting team, and the code is open source here. I cloned a copy of their issue-tracker gadget under a littleware 'gadgets' repository, and was able to add a milestone parameter to the gadget with only a little bit of trouble. I think I'm all set!


I submitted an issue with the gadgets for google code project with some information about my patch. It would be cool if they fold the patch into their code - we'll see what happens.

Wednesday, February 23, 2011

Success tastes like pizza!

I think I finally have the blogger podcast thing working. To recap - I started by hosting my .mp3 media file with Google docs, and that works great, except Google docs publishes the .mp3 file to a URL that does not include the .mp3 file extension (something like https://docs.google.com/uc?id=0B-82pgvnWy-8OWM2ZWJhYzktOWU5OC00MTI4LWFiZWYtZWNhOTQ0ZjlmMDA2&export=download&hl=en), and iTunes requires that a podcast's enclosure references a URL with a valid media extension (more details here).

So I went looking for some other free place to host my .mp3 files, and found Microsoft's sky drive, which preserves a file's extension in its download URL (ex: http://public.blu.livefilestore.com/y1p5DNvAssqO8iaDVAKdFChzS01Pj_v3M9B68-VJdTj9sQr43GHTr8pThRglPvVeIzJlbU4AE3nQbW7NiocbkkOLA/shCore.js?download&psid=1). Unfortunately, that download URL does not persist - a file's URL changes from day to day.

Finally, I tried Google sites, and (so far) I think that works - with persistent iTunes-friendly URLs like https://sites.google.com/site/frickjack/projects/podcasts/Podcast20110223.mp3?attredirects=0&d=1

This picture shows our dog, Moose, guarding a pizza from our other dog, Ponzi - Moose wants it all for himself! One tip for home made pizza - if you don't have a pizza stone, then an upside down iron skillet works too.

From Food
Picasa logo

Skydrive lets me down!

Ugh!

I was pretty happy when I was able to setup a blogger based podcast with mp3 files served from Microsoft's sky drive, so I went ahead and linked my blogger template to a copy of Gorbachev's syntax highlighter on sky drive too, but it turns out the freakin' sky-drive URL's do not persist, so neither of those nifty tricks work. I just inlined the syntax highlighter into the blogger template, but I'll have to figure something else out for audio-file hosting. There must be some way to get Google Docs URL's to work with iTunes.

This is from Apple's "Making a Podcast" page:
The URL before the GET-style form values (before the first ?) must end in a media file extension (e.g. mp3). To work around this, the feed provider can alter their URL from this: http://www.podcaster.com/load.php?f=&Wipeout.php to this: http://www.podcaster.com/load.mp3?f=&Wipeout.mp3 Notice how it says load.mp3 instead of load.php. It should be possible to accomplish this via various means, such as web server rewrites. iTunes looks at the extension of the path part of the url, i.e. the part before the"?".

Ugh!

JScrollPane a JPanel !

If you're a frickjack like me, then you might believe that a JScrollPane only works with widgets like JTextArea that are specially designed to be scrollable, but (as with so many other beliefs) you would be wrong! Turns out JScrollPane works just fine with JPanel, which is a big deal when you're trying to layout a bunch of widgets on a big form or whatever. Anyway, the scala code below works, so I hope scrolling works for more complicated panels too. I'll let you know if I find out otherwise.

package littleware.demo

import javax.swing._

object Toy {
    def main( args:Array[String] ):Unit = {
        SwingUtilities.invokeLater( new Toy )
    }
}

class Toy extends Runnable {
    override def run():Unit = {
        val jbox = new Box( BoxLayout.Y_AXIS )
        (0 until 40).foreach( (index) => {
            jbox.add( new JLabel( "Test" + index ) )
        }
        )
        val jframe = new JFrame( "Toy" )
        jframe.setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE )
        val scroll = new JScrollPane( jbox )
        scroll.setVerticalScrollBarPolicy( ScrollPaneConstants.VERTICAL_SCROLLBAR_ALWAYS )
        scroll.setPreferredSize( new java.awt.Dimension( 700, 400 ))
        jframe.add( scroll )
        jframe.pack
        jframe.setVisible( true )
    }
}

How do you like that slick syntax highlighting ? If it works (I won't know till I post, so frick), then it's all thanks to Alex Gorbachev's syntax highlighter - which I copied over to Skydrive, and hooked into the bLog template. Alex is probably an evil James Bond villain, but I'll have to paypal him $10- when I get motivated.

That's all pretty exciting stuff. Our dogs were very excited too ...

the pack is bored

Friday, February 18, 2011

Pound cake!

 

What to do with 1/2 pound butter, 5/3 cups sugar, 2 cups flour, ... Good old Fannie Farmer!

 

I also accidentally uploaded this pretty cool picture of my dog, Shug, while playing with Picasa. Go, Shug!

Posted by Picasa

Thursday, February 17, 2011

Blogger + Google Docs Podcast

I often enjoy listening to podcasts while I'm playing around with the computer. The java posse, TED, Entrepreneurial Thought Leaders, and the New Yorker Political Scene are some of my favorites.

I recently began wondering how I could publish my own podcast of (for example) a lecture at the university or whatever with as little expense and effort as possible. It turns out that cheap and easy podcast publishing can be a simple 3 step process thanks to great free online services like Google's docs and blogger, and the very useful open source audacity audio program (or Garage Band is probably even better on the Mac).

  • First, record your podcast with audacity or garage band or your iPhone or whatever. You can get as professional as you want - I'm just a hack for now.
  • Second, publish your audio file online. Google docs gives you 1 GB of storage for free, but Microsoft's skydrive provides 25 GB for free!
  • Third, setup a podcast feed that links to your audio file. A podcast feed is just a specialized RSS feed, which is just an XML file that readers know how to interpret. You could just edit an rss xml file, and publish that via Google docs or skydrive, but Google's blogger platform makes it easy to associate audio files with blog entries, so the blog feed also acts as a podcast feed.
  • Finally, register the podcast's feed url (ex: http://blog.frickjack.com/feeds/posts/default?alt=rss) with iTunes or your favorite podcast player.

Even an idiot like me can pollute the internet with worthless media!

Update ...

I wrote this post before I actually had the podcast feed working, and I wound up driving myself crazy trying to get iTunes to recognize the podcast when hosting the .mp3 file on Google docs. The feed worked fine with Google Reader, but I think iTunes looks at the enclosure URL to decide what kind of media file a feed item references rather than trusting the enclosure's mime type attribute. Google docs generates nonsensical urls that do not preserve the file extension (ex: https://docs.google.com/uc?id=0B-82pgvnWy-8OWM2ZWJhYzktOWU5OC00MTI4LWFiZWYtZWNhOTQ0ZjlmMDA2&export=download&hl=en is a .mp3 file), so iTunes throws a
There are no playable episodes ... The URL might point to text-only episodes, or contain file types that iTunes cannot play.
error when loading the podcast.

Fortunately, Skydrive came to my rescue with an iTunes friendly URL for my audio file: http://m8cb3q.blu.livefilestore.com/y1pCrRuW5NqZivMGgB7_oAUy_g8m_83domL0-7pAkoVezoGWOATDPuuLQNxECsNAK60JSxlYwkgHW2WycExfTMGXBxd4-gM0bqR/Podcast20110217.mp3?download&psid=1.

It also turns out that Google Reader is nice about letting random frickjacks embed its audio player!

Another Update ...

Freakin' sky-drive url's don't persist (there's some kind of cache timeout mojo), so the .mp3 links are now bogus. Back to the drawing board!

Wednesday, February 09, 2011

Apache mod_proxy build trickery

I lost a few more hairs building mod_proxy for an existing Apache install. It turns out mod_proxy requires proxy_util.c on the apxs build line.

   apxs -i -c mod_proxy.c  proxy_util.c

Of course the .so builds and installs fine without proxy_util.c, so you think everything is fine until apache pukes with a "missing symbol" error. Frickjacks!

Friday, December 03, 2010

Chrome RSS extension

Before today when I wanted to add a web site's feed to Google Reader from Chrome I would "view source", find the feed URL, then go to google reader to subscribe to the feed. Today I finally got a brain, and checked to see if there's a chrome extension that automates this process, and I found exactly what I wanted. . It's great when someone solves a problem for me!

Tuesday, November 23, 2010

Glassfish JDBCRealm authentication against DSpace user database

A project we're working on at the library requires a DSpace repository webapp to cooperate with a custom data-submission webapp that we're building. We want our custom webapp to authenticate with the same credentials that dspace manages in its internal database. Luckily, our glassfish app-server includes a JDBC security realm that fit this problem perfectly.

It took me several hours of head banging and googling ( http://blogs.sun.com/swchan/entry/jdbcrealm_in_glassfish_with_mysql , http://blogs.sun.com/swchan/entry/jdbcrealm_in_glassfish , http://blogs.sun.com/swchan/entry/assign_groups , http://stackoverflow.com/questions/719708/java-web-application-using-a-custom-realm ) to figure out all the steps to properly configure a glassfish JDBCRealm to authenticate our webapp. I was really glad it worked in the end. I configured a glassfish JDBC security realms with the following properties to access the 'email' and 'password' column's in the dspace 'eperson' database table. The following data in domain.xml can be configured via the glassfish admin webapp

    <jdbc-connection-pool
datasource-classname="org.postgresql.ds.PGSimpleDataSource"
is-isolation-level-guaranteed="false" res-type="javax.sql.DataSource"
description="connection to local dspace database" name="dspace">
      <property name="DatabaseName" value="dspace" />
      <property name="LogLevel" value="0" />
      <property name="LoginTimeout" value="0" />
      <property name="Password" value="XXXXX" />
      <property name="PortNumber" value="0" />
      <property name="PrepareThreshold" value="5" />
      <property name="ProtocolVersion" value="0" />
      <property name="ServerName" value="localhost" />
      <property name="SocketTimeout" value="0" />
      <property name="Ssl" value="false" />
      <property name="TcpKeepAlive" value="false" />
      <property name="UnknownLength" value="2147483647" />
      <property name="User" value="XXXXX" />
    </jdbc-connection-pool>
 
    <jdbc-resource pool-name="dspace" description="dspace data source"
jndi-name="jdbc/dspace" />
 
        <auth-realm
classname="com.sun.enterprise.security.auth.realm.jdbc.JDBCRealm"
name="dspaceRealm">
          <property name="datasource-jndi" value="jdbc/dspace" />
          <property name="jaas-context" value="jdbcRealm" />
          <property name="user-table" value="eperson" />
          <property name="user-name-column" value="email" />
          <property name="password-column" value="password" />
          <property name="group-table" value="epersongroup" />
          <property name="group-name-column" value="name" />
          <property name="assign-groups" value="users" />
        </auth-realm>
 

Finally, we configure our webapp to use the dspaceRealm with the following blocks in WEB-INF/web.xml:

 
    <!--
       Login config for production server.
       See role-mapping in sun-web.xml too.
       Details of dspaceRealm configuration for glassfish in bLog.
 
    <login-config>
        <auth-method>BASIC</auth-method>
        <realm-name>dspaceRealm</realm-name>
    </login-config>
    <security-role>
        <role-name>users</role-name>
    </security-role>
    <security-constraint>
        <web-resource-collection>
            <web-resource-name>Secure Pages</web-resource-name>
            <url-pattern>/*</url-pattern>
            <http-method>GET</http-method>
            <http-method>POST</http-method>
        </web-resource-collection>
        <auth-constraint>
            <role-name>users</role-name>
        </auth-constraint>
    </security-constraint>
   -->

... and WEB-INF/sun-web.xml:

    <!-- Enable on production server - see note in web.xml ....
    <security-role-mapping>
        <role-name>users</role-name>
        <group-name>users</group-name>
    </security-role-mapping>
    -->

In the end I wanted to send an e-mail to the dspace tech-list, but I was disappointed to discover that there is not a standard way to incorporate JAAS authentication into an application server security realm. All the app servers have a slightly different way of implementing security realms, and the setup I describe above is glassfish specific, but that's good enough for me for now.

Saturday, October 02, 2010

Too Big To Fail

Last week I finished reading Andrew Sorkin's Too Big To Fail. It was a great read! I was engrossed by the story of the events behind the financial system's near collapse in September, 2008, and the handful of personalities that worked to save the system. The book especially gives insight into the monumental bad decision to allow Lehman Brothers to go bankrupt, the acquisition or legal transition of the other big investment banks, the AIG mess, the passage of TARP into law, and the government's subsequent capital injection into U.S. banks in exchange for equity.

Sunday, September 26, 2010

Powershell browse and recycle functions

I just added browse and recycle functions to my powershell profile: Documents/WindowsPowershell/profile.ps1. The browse function opens up an explorer window at a specified path, and the recycle function sends a path to the recycle-bin after prompting for user confirmation.

# Excerpt from profile.ps1
...
$GLOBAL:docs = $home + "\Documents"
$GLOBAL:trash = $Env:TEMP
$GLOBAL:shell = new-object -com Shell.Application
$Env:nodosfilewarning=1
...
function recycle( $path ) {
    $clean = resolve-path( $path )
    $item = $shell.Namespace(0).ParseName( $clean )
    $item.InvokeVerb("delete") 
}

function browse( $path ) {
    $clean = resolve-path( $path )
    #echo $clean.path
    $shell.open( $clean.path ) 
}

Sunday, September 12, 2010

Simple Parallelism with Scala

Scala provides some great list processing API's, and it's easy to parallelize Scala's list processing with a java executor service. The following demonstration shows a couple easy ways to parallelize list processing. The second by2Groupwise variation processes chunks of a work list in parallel, and reports progress to the user between chunks. Both by2Groupwise and by2Parallel follow a fork-join pattern where a pass over the work list submits callable tasks to an executor, then a subsequent pass waits for the results.


import java.util.Date
import java.util.concurrent.{Callable,Executors,ExecutorService}
import java.util.logging.{Level,Logger}


object Demo {
    val log = Logger.getLogger( getClass.getName )
    
    implicit def callable[A]( func:() => A ): Callable[A] = new Callable[A] {
         override def call:A = func()
    }
    
    
    val poolSize = 4
    val exec = Executors.newFixedThreadPool( poolSize )
    
    var lastReport = -1;
    def reportProgress( soFar:Int, total:Int ):Unit = {
        val progress = (soFar * 100) / total
        if ( progress != lastReport ) {
            log.log( Level.INFO, "Progress: {0}%", Array[Object]( progress.toString ) )
            lastReport = progress
        }
    }
    
    def timer[A]( work: => A ):(Long,A) = {
        val start = new Date
        val result = work
        ( (new Date).getTime - start.getTime, result )
    }
    
    def by2( arg:Int ):Int = {
         Thread.sleep( 1000 )
         arg * 2     
    }
    
    def by2Sequential( work:Seq[Int] ):Seq[Int] = work.zipWithIndex.map( 
         _ match { case (arg, index) => { 
             reportProgress( index, work.size )
             by2(arg)
           } } )
    
    def by2Parallel( work:Seq[Int] ):Seq[Int] = {
         work.map( (arg) => exec.submit( () => by2(arg) )
                ).map( (future) => future.get )
    }
    
    def by2Groupwise( work:Seq[Int] ):Seq[Int] = {
        val groups = work.grouped( poolSize ).toList
        log.log( Level.FINE, "Frickjack size: {0}", Array[Object]( groups.size.toString ) )
        groups.zipWithIndex.map( 
          _ match { 
            case (group,index) => {
              log.log( Level.INFO, "Scan index: {0}", index )
              reportProgress( index, groups.size )
              group.map( (arg) => exec.submit( () => by2(arg) )
                     ).map( (future) => future.get )
            }
            }
        ).toList.flatten
    }
    
    def main( args:Array[String] ):Unit = {
        val work = 1 until 10
        log.log( Level.INFO, "Seq {0}", Array[Object]( timer( by2Sequential( work ) )._1.toString ) )
        log.log( Level.INFO, "Parallel {0}", Array[Object]( timer( by2Parallel( work ) )._1.toString ) )
        log.log( Level.INFO, "Groupwise {0}", Array[Object]( timer( by2Groupwise( work ) )._1.toString ) )
        exec.shutdown
    }
}

Saturday, August 21, 2010

frickjack.com DNS forwarding to Google Sites

Today I finally wired up my frickjack.com domain configuration with GoDaddy to just redirect (via domain forwarding) to pages hosted on Google Sites. I bought the frickjack.com domain a few years ago with the idea that I'd get a static IP address, and run a web server from home with a bLog, code repostiory, and some demonstration littleware web applications.

I actually did run my own server on an old laptop for a while, but I always felt bad leaving my old laptop running in the corner just to host a site no one looked at anyway. I tried to manage my own bLog with XML and XSL, but Blogger and other hosted solutions are free and easy to use. Similarly, I've been very happy letting Google Code host my littleware project.

Anyway, Google Sites gives me an easy way to put frickjack.com back online as the root node of the graph that connects my disembodied web presence across blogger, facebook, google code, and other sites.

Sunday, June 13, 2010

Bootstrap Bonanza

This post is a copy of this google doc describing littleware's bootstrap process.

I recently reworked the littleware bootstrap and dependency injection process. In the new system Guice manages dependency injection; java's ServiceLoader utility discovers the littleware modules available in the classpath, and OSGi manages startup and shutdown of subsystems. The new bootstrap code also supports different runtime profiles similar in spirit to java EE6 profiles or configurations in Apache Ivy. The new code both simplifies the process an application follows to configure, startup, and shutdown a littleware application; and also simplifies the artifacts that a littleware extension module must provide to register itself with littleware's runtime to expose its functionality to the application.

Littleware has a multi-step bootstrap process. The parent application determines whether to start littleware in a "server", "client", or "app" configuration. With the "client" configuration littleware's bootstrap establishes and authenticates a user session with a remote littleware server (which is another application running a "server" configuration). The "app" configuration allows applications that do not interact with a littleware server to still take advantage of littleware tools that support generic standalone GUI and CLI applications.

The "server", "client", and "app" systems have similar bootstrap processes. A "client" application begins by allocating a ClientBootstrap.ClientBuilder type object:

       final ClientBootstrap.ClientBuilder builder =           
             ClientBootstrap.clientProvider.get();

The ClientBuilder automatically loads the ClientModuleFactory service providers registered in the application class path, and gives the application the opportunity to add or remove modules from the bootstrap module set. The ClientBuilder also allows the application to specify a a profile from the AppBootstrap.AppProfile enumeration. The available profiles include AppProfile.SwingApp (the default), AppProfile.JNDI, AppProfile.CliApp, and AppProfile.WebApp.

The ClientBuilder constructs a LoginSetup object that provides methods to manage user authentication including an automatic method that prompts the user for his credentials in a way appropriate to the AppProfile, and returns a ClientBootstrap object. The ClientBootstrap bootstrap method injects and returns an application supplied type.

        final ClientBootstrap boot = 
               builder.addModuleFactory( new MyModuleFactory() 
                         ).profile( AppBootstrap.AppProfile.CliApp 
                         ).automatic();
        final MyTool tool = boot.bootstrap(MyTool.class);

The littleware runtime is up and running at this point. The MyTool class should be annotated so Guice injects whatever dependences the application needs.

         class MyTool {
                 ...
                   @Inject
                    public MyTool( AssetSearchManager search ) {
                           this.search = search;
                    }
                 ...

Under the bootstrap hood ClientBootstrap loops over the ClientModuleFactory service providers to build a ClientModule (a Guice Module subtype) configured :

         factory.build( profile )

ClientBootstrap builds a Guice Injector configured by the resulting collection of ClientModule objects, then loops over the ClientModule collection again to inject the OSGi BundleActivator associated with each module. Next ClientBootstrap launches an OSGi runtime to startup the activators. Finally, ClientBootstrap injects and returns MyTool.class the application passed to the bootstrap() method.

Saturday, May 15, 2010

Stacked Authentication with JAAS

I was happy the other day to find a quick solution to an authentication problem using java's authentication and authorization service ( JAAS). I was configuring JAAS authentication for a littleware based webapp where I wanted most users to authenticate via active directory, but to also have a simple password file to manage authentication for some internal users like "guest". Fortunately, JAAS easily supports this kind of thing. After googling around I stumbled across the JMX com.sun.jmx.remote.security.FileLoginModule which uses a simple password file, and I was already familiar with using the com.sun.security.auth.module.LdapLoginModule for simple authentication against Active Directory. I simply configured the "littleware.login" context in the Glassfish server domain's login.conf file (below) so that authentication via either A.D. LDAP bind or the password file would both be sufficient to certify a user. Works like a charm!

littleware.login {
        com.sun.security.auth.module.LdapLoginModule SUFFICIENT
             userProvider="ldap://xxxxxx/dc=auburn,dc=edu"
             authIdentity="{USERNAME}@auburn.edu"
             userFilter="(cn:dn:={USERNAME})"
             useSSL=false
             debug=true;
        com.sun.jmx.remote.security.FileLoginModule SUFFICIENT
             passwordFile="/home/glassfish/.littleware/jaasPasswordFile.txt";
};

Tuesday, April 20, 2010

Scala dependent method type

Scala dependent type exception

I'm building a simple JSF-3 web application that implements backing beans in Scala 2.8 One bean includes Enumeration properties that a JSF web-form exposes to clients via a combo-box control configured via a key-value map.

    <h:selectOneMenu id="patronType" value="#{sampleBean.patronType}">
        <f:selectItems value="#{sampleBean.patronTypes}" />
    </h:selectOneMenu>

The sampleBean's definition includes patronType and paronTypes properties which are Enumeration subtypes.

object PatronType extends Enumeration {
  val Student, Faculty, Other = Value
}

...

@ManagedBean
@RequestScoped
class SampleBean extends InjectMeBean {
  private def enumToMap( enum:Enumeration ):java.util.Map[String,Enumeration#Value] =
        ((new ImmutableMap.Builder[String,Enumeration#Value]) /: enum.values) (
      (builder,member) => builder.put( member.toString, member )
    ).build

...
  @reflect.BeanProperty
  var patronType:PatronType.Value = PatronType.Student

  @reflect.BeanProperty
  val patronTypes:java.util.Map[String,Enumeration#Value] = enumToMap( PatronType )
}

I originally defined the patronTypes property as a mapping from String to PatronType.Value.

  @reflect.BeanProperty
  val patronTypes:java.util.Map[String,PatronType.Value] =
               ((new ImmutableMap.Builder[String,PatronType.Value]) /: enum.values) (
        (builder,member) => builder.put( member.toString, member )
      ).build

I introduced the enumToMap method to share that code block between several similar properties with different Enumeration types. Unfortunately - a subtype-specific enumToMap implementation has on output type that depends on an input parameter.

scala> def enumToMap( enum:Enumeration ):java.util.Map[String,enum.Value] =
     |         ((new ImmutableMap.Builder[String,enum.Value]) /: enum.values) (
     |       (builder,member) => builder.put( member.toString, member )
     |     ).build
<console>:6: error: illegal dependent method type
       def enumToMap( enum:Enumeration ):java.util.Map[String,enum.Value] =
                      ^    

I tried several variations on this theme, but finally settled on the enumToMap implementation that just leverages the Enumeration#Value base type. I googled around while working on this code, and stumbled across the fact that scala includes limited experimental support for dependent method types - that will be very cool if it evolves to production status.

Scala's instance-attached nested types have advantages, but they feel alien to an experienced java developer like me.

scala> class A {
     | var a = "A"
     | class B {
     | var b = "B"
     | }
     | var b = new B
     | }
defined class A

scala> val a1 = new A
a1: A = A@6489f0

scala> val a2 = new A
a2: A = A@15c44d6

scala> a1.b = a2.b
<console>:9: error: type mismatch;
 found   : a2.B
 required: a1.B
       a1.b = a2.b
                 ^

Friday, March 26, 2010

Wisdom

I sometimes find myself thinking as I look back at life "Man! I was an idiot back then!" where "back then" might be ten years ago or last year or yesterday or the previous paragraph.

As time passes, I'll think again, "I was an idiot back then" remembering my thoughts the last time I thought what an idiot I was. Eventually I realize that even as I'm thinking what an idiot I was in the past, that in the future I'll look back on the present and think, "What an idiot!".

In time I'll think "I was an idiot when I thought I was an idiot before, because what I did before was not so idiotic although I was too much of an idiot to realize it at the time" - sort of like multiplying two negatives. Of course it's possible to multiply three negatives, "I was an idiot when I thought I was an idiot for thinking I was an idiot".

I know I'll eventually find myself in some bed after the ravages of parkinsons, alzheimers, and multiple strokes leave me with half my body frozen in place and the other half constantly shaking violently. I'll be thinking, "Where am I ? Who is this person feeding me oatmeal ? Who am I ? This oatmeal tastes good! Where am I ? What is this in my mouth ? It tastes good!".

Sunday, February 14, 2010

Larry Ellison in the Cloud

I enjoyed the webcasts Oracle published to outline its product strategy upon the approval of Oracle's acquisition of Sun Microsystems. I was impressed with most of the ideas presented, and I think Oracle will have some success selling their integrated stack "from disk to whatever". However, I was surprised by Larry Ellison's criticism of "cloud computing" when responding to a question near the very end of the event.

Cloud service providers like Amazon EC2 allow a software service developer to deploy one or more copies of a virtual machine image that provides some online software service. A "cloud" service may be a natural evolution of standard hosting provider, but the ability to dynamically allocate and deploy compute resources opens doors for a "software as a service" (Saas) developer. A developer of some service can quickly provision and deploy a virtual server dedicated to some new customer, or scale an application horizontally or vertically in response to increased client load.

A developer team that codes for the cloud avoids the cost and complexity of maintaining servers, networking switches, or storage in a data center. Managing data center complexity is where Oracle makes a lot of its money. Furthermore, the cloud is a natural platform for free open technologies like LAMP that are free to the developer, and do not provide a clear business model for a technology infrastructure companies like Oracle.

Oracle has tremendous breadth and depth of resources, and I don't think Larry Ellison's cloud myopia will be a problem of any importance, but it's strange when a technology leader comes late to an important idea.