Let’s say we want to compress a directory, transfer it over the network, then decompress it onto our current machine, and do the whole thing without creating temporary files on either system. Here’s one way to do it:

sudo ssh -p 2222 user@host 'cd /home/user/ ; tar -c "iwant/" | bzip2 -9 -c' \
| bunzip2 -c | sudo tar -x

Here’s how it works:

1. sudo invokes ssh (you may need to enter a password).
2. SSH connects to the host (server) on port 2222 and authenticates. (You may need to enter a password). A shell is now open on the host.
3. The shell on the host changes its working directory to /home/user.
4. Tar begins to compress the files in the iwant directory and sends its output to STDOUT.
5. bzip2 compresses the data from STDIN (sent by tar) and sends it to STDOUT
6. The data from STDOUT (sent by bzip2) goes across the network to the machine that initiated the SSH connection and to that process’ STDOUT.
7. ssh’s STDOUT is piped to bunzip2′s STDIN. bunzip2 then writes decompressed data to STDOUT.
8. sudo starts up tar, which reads from STDIN (uncompressed data from bunzip2) and recreates the directory structure, file names, and file content.

To be more concise, the sequence is: connect, tar, compress, network transmit, uncompress, untar, disconnect. The first sudo call is probably unnecessary, but the second one is required if you lack write permission to your current directory.

The quickest way to test this is to ssh to localhost. Getting the command exactly right can be tricky, so if you’re trying to transfer large files this way, it’s a good idea to test the command locally first.

So, there you have it: compressed data transfers without temp files. Whether or not you get faster transfer speeds is another story. Each situation is different. You can find the answer to that on your own.

But when you’re one of the largest Internet Service Providers in the US, lack of resources probably isn’t the cause of awful customer service experiences. While I could try to be fair and track down some real, underlying cause to my present frustration, I will go the simple route and claim institutional malevolence. It’s more satisfying, and probably more accurate anyway.

Here’s the problem: the billing service I’ve been using–you know, the system they use to get money from me every month–is being upgraded or migrated or whatever. Yep, the old PayXpress^SFTM system won’t be accepting its own logins anymore, so now customers will all use  something they (so cleverly) call MyServices^RFSFTM. But they never notified me that I was going to be forced into this new system. And they did nothing to prepare me for it, much less to make it easy.

Let me repeat myself: I received no notices of this migration. Not in the mail, not a phone call, not an email. I think I’m entitled to at least one email for an event like this, but maybe my expectations are just too high. But this wasn’t even mentioned in the monthly “newsletter” email they send me (that I filter and never read because it’s full of dumb, useless crap). Silence. That’s it.

So where was the problem? Well, in their eagerness to migrate my account to the new system, they are actively preventing me from getting into the old system. That’s pretty much the definition of malevolent, if you ask me. From a user’s perspective, it is disorienting, frustrating, and time-wasting. And wasting a customer’s time is not amoral. Treating customers in this manner cannot be for technical reasons: the old login still works. The customer’s experience is simply not a priority: the needs of the bureaucracy come first.

Now I’ve been through the friendly version of this type of activity with other companies, and it doesn’t have to be so bad. It goes like this: log in like you always do, and do what you always do; when you’re ready, or you’re out of time, follow a link to complete the migration steps. In this case, I still have more than 30 days before the migration deadline, so it’s not like I’ve been procrastinating. I’m practically being forced to migrate a month ahead of their own schedule, with utter and complete disregard for my own schedule.

So what made this process so painful and obnoxious that it deserves a blog post? Well, they decided that I needed to pointlessly re-enter data such as my “account number” and “customer code.” Information that I–and this may come as a surprise to the idiots who planned this migration–actually don’t have on hand. Even though I’ve had this account for over a year and a half. I just can’t get motivated to memorize an account number I’ve looked at just once in 18 months.

Why don’t I have these numbers on hand? Because I use paperless billing. I always have. And that information is only found on my billing statement. Which is only online. (Really.) Which I can only access through the old PayXpress^SFTM system. (Seriously.) The same system whose login page is being actively and maliciously hidden from me.

Now remember this is a migration: they already have all of my personal information: name, address, username, password, etc. Forcing me to re-enter information that has already been verified is a waste of my time. It’s disrespect for my time. Especially when it’s information you made up yourself in the first place.

Entering this data certainly doesn’t make the migration process more “secure,” but I’d bet they’d be willing to try that lie out if they thought they could get away with it. Obviously if I have a working username and password, which I can use to log in and look up the information you’re asking for, there is no added security involved in my pointlessly reentering that same information.

And honestly, who keeps a copy of their cable bill within easy reach at all times? News flash Time Warner Cable: your customers don’t love you that much! Quite the opposite in my case: I really have less than no desire to go searching my house for a piece of paper with your logo on it. I’ve got better things to do with my time. It makes me hate you even more than I already do.

So how did I solve this little catch-22? Well, I did manage to find the login page for the old system, but it’s not on any of the pages that try to “help” you through the process of getting onto the new system. Once I did get in, I looked up an old billing statement and found the information they wanted. Which they already had. Obviously. Whatever. And just to make a point, I paid my bill one last time using the old system before migrating to the new one. But it’s still really stupid to force me to enter data that’s already stored on your system. And I’m still irritated about this whole process. In a world of ETL and web services, such things are inexcusable.

There is a workaround, though the documentation isn’t the greatest. It involves using a program called ld-linux.so. (It’s not in /bin, but try `man ld-linux.so` for documentation.) I should mention I’m on a 64-bit installation, as that affects the commands I use. Here’s more or less the steps to take:

1. Download pre-built binaries of neon-0.28.4-1.x86_64.rpm and subversion-1.6.16-1.rhel5.x86_64.rpm. I used wget, but curl or some other tool is fine as well.
2. Convert the rpm files to cpio files (rpm2cpio package.rpm > package.cpio)
3. Extract the files through a process of trial and error. This means running `cpio -i < package.cpio` and then looking through the errors and running `mkdir -p /missing/cpio/path` as many times as is needed.
4. Move the various files to my home directory (e.g. /home/me/usr/lib64, /home/me/usr/bin).
5. Create some variation of this script, one for each binary you want to run:

#!/bin/sh
/lib64/ld-linux-x86-64.so.2 --library-path '/home/me/usr/lib64:/usr/lib64:/lib64' ~/usr/bin/svn $*

And that’s enough to make it work! To run a 32-bit app, use something like /lib/ld-linux.so.2 instead.

Alternatively, you can add LD_LIBRARY_PATH to your .profile. That’s supposedly the Wrong Way™ to do it, but it will work, and it would eliminate the need for creating scripts and calling ld-linux.so. Note carefully that you must use ‘export’ when setting LD_LIBRARY_PATH. I didn’t realize this until I saw someone’s working example. Like I said, the documentation isn’t the greatest. Something like this:

LIBRARY32=/lib:/usr/lib:/usr/share/lib
export LD_LIBRARY_PATH=/home/me/usr/lib64:/usr/lib64:/lib64:$LIBRARY32

Not being able to install shared libraries makes life difficult. Sure it’s nice that I can run `ldd ~/usr/bin/svn |grep 'not found'` but that just tells me what’s missing. What would be really nice is a way to change the location of /etc/ld.so.cache for my user account’s environment. I certainly haven’t found any evidence that such a thing is possible, but it would be nice. Indeed, that seems like a fairly useless parameter for ldconfig to support except for that rare person who is just trying to test ldconfig itself.

So there you have it: a way to install Subversion when you lack root access and are missing required libraries. It’s actually fairly easy, once you know what to do. Much of the process could be automated with some shell scripts, but I’m not motivated enough to spend an hour or two writing them.

<complexType name="SpeciaRepeatingType">
	<sequence minOccurs=1" maxOccurs="unbounded">
		<element name="RepeatValue" type="custom:RepeatType" />
	</sequence>
</complexType>

I used this idiom in several places, which turned out to be a huge problem. The wsdl2java tool I was using just couldn’t handle this markup. Here is why: the schema doesn’t actually have a repeating element of type RepeatType as I intended. In actuality it has a repeating sequence, and that sequence has only one element in it—of type RepeatType.

While that does not excuse the bad behavior of wsdl2java, it does not mean that I had expressed my intent correctly. What I really wanted was this:

<complexType name="SpeciaRepeatingType">
	<sequence>
		<element name="RepeatValue" type="custom:RepeatType"
			minOccurs=1" maxOccurs="unbounded" />
	</sequence>
</complexType>

So there you have the correct way to do it: minOccurs and maxOccurs should typically be used on elements, not on sequences. Especially if you are using some WSDL parsing tool. It’s that simple.

Another lesson to learn is this: even if Eclipse can generate schema-compliant test documents, you can still have problems with your WSDL. Interoperability is more than just a valid XML Schema.

<input type="button" class="iceCmdBtn " value="delete"
    onclick="iceSubmitPartial(form, this, event);return false;" />

Now, I’m not an expert in CSS, but in the last 2 years I’ve worked with 2 very good designers, and both have complained about this behavior. The most recent complaint came on the heels of a bunch of JSF calendar rendering work I had been doing, so I am now the first person in my company to have experience with the low-level details needed to make this work. So within 15 minutes I had a solution put together. In fact, the hardest part was to figure out what to put in the configuration files.

In my project’s WEB-INF/faces-config.xml (based on what I found in /icefaces/component/conf/META-INF/faces-config.xml):

<render-kit>
    <render-kit-id>ICEfacesRenderKit</render-kit-id>
    <render-kit-class>com.icesoft.faces.renderkit.D2DRenderKit</render-kit-class>
    <renderer>
        <component-family>javax.faces.Command</component-family>
        <renderer-type>com.icesoft.faces.Button</renderer-type>
        <renderer-class>com.inco5.jsf.ice.renderkit.ButtonRenderer</renderer-class>
    </renderer>
</render-kit>

Then, to actually get different output, I needed to extend the existing ICEfaces button renderer. My class file looked a bit like this:

package com.inco5.jsf.ice.renderkit;
public class ButtonRenderer extends com.icesoft.faces.component.ext.renderkit.ButtonRenderer
    @Override
    public void encodeBegin(final FacesContext facesContext, final UIComponent uiComponent) throws IOException {
    …
    final Element root = domContext.createElement("button");
    …
    }

So there you have it: a proper <button> tag, and all the benefit that goes along with it. I don’t know what all the potential side-effects of the change might be, but it is nice to know how easy it can be to alter the behavior of a tool that doesn’t always cooperate in ways we would like.
2011-01-18 Update — Here is a markup sample:

<button class="iceCmdBtn myButtonClass"
    id="js_01_:myform:mybutton" name="js_01_myform:mybutton"
    onclick="iceSubmitPartial(form, this, event);return false;"
><span>My Button</span></button>

This particular case had me up until midnight with nothing but frustration to show for it. After a bit of sleep (one of the key elements of software debugging that is often overlooked) it dawned on me what my problem was. Consider the code:

public Class MyBean
{
    enum MyEnum {
        VAR1,
        VAR2;
        public String getTitle() {
            return name().toLowerCase();
        }
    }
    …
}

And the related facelet markup:

#{myEnumValue.class} is titled '#{myEnumValue.title}'

It seemed to be straightforward, but I kept getting this error message:

Property 'title' not readable on type java.lang.String

And what kept bugging me was the fact that #{myEnumValue.class} worked when it was by itself (it would output MyBean.MyEnum). I could clearly see the class name, and it looked just fine. But as soon as I added in the title property, I would get the error message again. Obviously myEnumValue was not a String, so why does the error message say that it is?

As it turns out, my Enum wasn’t public, and so apparently the methods on it could not be invoked. Rather than telling me that, the expression evaluator decided to coerce the Enum into a String and then see if the property I wanted was part of the String class. It wasn’t. Here’s the solution:

public Class MyBean
{
    public enum MyEnum {
    …
    }
}

So there you have it: one more strange error message deciphered. I hope others will spend less time tracking down these types of errors than I do.

And the real moral of the story? Well, first, error handling is hard. While string coercion may be a good strategy for this situation, the fallback error message proves unhelpful. Better to capture the message that triggered the fallback and return it instead. Unfortunately, that requires that your error handling strategy can maintain such a message and know when to use it. So a second moral is, be prepared for error handling to get much more complex as your requirements expand.

Beyond that, you’ll have to wait for a future article where I will explain in more detail the design flaw that makes this particular error possible. I will update this post to point to the new article when it is complete.

I figured this out by accident when I was trying to fix the following error message produced by archetype:generate:

[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error building POM (may not be this project's POM).

Project ID: unknown

Reason: Could not find the model file '/home/user/projects/test/newsubproject'. for project unknown

[INFO] ------------------------------------------------------------------------
[INFO] Trace
org.apache.maven.reactor.MavenExecutionException: Could not find the model file '/home/user/projects/test/newsubproject'. for project unknown
at org.apache.maven.DefaultMaven.getProjects(DefaultMaven.java:378)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:292)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:129)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:287)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.apache.maven.project.ProjectBuildingException: Could not find the model file '/home/user/projects/test/newsubproject'. for project unknown
at org.apache.maven.project.DefaultMavenProjectBuilder.readModel(DefaultMavenProjectBuilder.java:1557)
at org.apache.maven.project.DefaultMavenProjectBuilder.buildFromSourceFileInternal(DefaultMavenProjectBuilder.java:504)
at org.apache.maven.project.DefaultMavenProjectBuilder.build(DefaultMavenProjectBuilder.java:198)
at org.apache.maven.DefaultMaven.getProject(DefaultMaven.java:583)
at org.apache.maven.DefaultMaven.collectProjects(DefaultMaven.java:461)
at org.apache.maven.DefaultMaven.collectProjects(DefaultMaven.java:534)
at org.apache.maven.DefaultMaven.getProjects(DefaultMaven.java:365)
... 11 more
Caused by: java.io.FileNotFoundException: /home/user/projects/test/newsubproject (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(FileInputStream.java:106)
at hidden.org.codehaus.plexus.util.xml.XmlReader.(XmlReader.java:123)
at hidden.org.codehaus.plexus.util.xml.XmlStreamReader.(XmlStreamReader.java:67)
at hidden.org.codehaus.plexus.util.ReaderFactory.newXmlReader(ReaderFactory.java:113)
at org.apache.maven.project.DefaultMavenProjectBuilder.readModel(DefaultMavenProjectBuilder.java:1552)
... 17 more

Incidentally, the solution to that problem was to rename the pom.xml file in the directory I was executing the command in (/home/user/projects/test/pom.xml). Apparently it’s not possible generate a project inside another project. The error message is entirely unintuitive, but you probably expected that if you’re a Maven veteran.

So there are two quick, helpful Maven tips that I hope will help you as much as it helped me.

Footnote:
I should mention that I was helped by Antony Stubbs on Stack Overflow. He had responded to a post from Hardik Mehta, who also wrote about it on his cout blog. Thanks guys.

Finally, I stumbled upon the solution, which I should have thought of in the first place. VirtualDub. It took a little coaxing to get the audio capture to work (I ended up selecting the USB webcam option) and the audio was out of sync so I had to adjust it in post processing (I set one file to -1100ms, the other -1550ms). All in all, though, it worked pretty well with minimal hassle. Yes, VirtualDub’s UI isn’t the most friendly, but it works, and I’m happy about that. An amazing piece of open source software.

More precisely, before working on a given transition, first complete all the conditionals required for it. Do this for each transition. If there are a number of transitions, it doesn’t necessarily make sense to work out all the conditionals for every transition before building any of the transitions, but it does make sense to approach it iteratively. Automated testing can be very helpful, since you can write unit tests for the conditionals, then once those pass you can test the transition results.