Role-based thinking – an experiment
I have previously written about the importance of thinking in terms of functional behavior rather than merely in terms of capability. That is, why we are doing something is at least as important as how we are doing it. Now, maybe I’ve been overdoing the whole Scrum thing lately (not likely), but I’ve started thinking more in terms of roles when I formulate tasks for my team. My old way of thinking (which I still frequently fall back into) goes something like this:
We have a lot of technical debt. We need to remove unused dependencies from our build. We need to improve our test coverage. We need to improve our release process. We need to be able to onboard new team members more easily. We need a more robust continuous integration environment. We need to make troubleshooting easier for production support. We need to fully externalize our application configuration. We need to minimize the impact of poor change management on the part of our upstream dependencies….
There’s nothing wrong with working through a task list such as this, but it does have shortcomings. First, it (often) leads us to prioritize personal preference over team productivity. When each team member pursues these types of activities in isolation, the outcome is not always generalizable and so the impact is dampened.
Second, it is difficult to justify these activities from a business perspective. The way I formulated the descriptions, there are no real quantifiable outcomes. While the objectives all sound good, we have to ask, what is the opportunity cost here? What are the tangible benefits to the team members who aren’t engineers? What are the benefits to the stakeholder the team is trying to serve?
Finally, these sorts of to-do lists are often very difficult to estimate against. How do we know when is a task done? Which parts of the task have highest priority? The answer, typically, is when the person who completes the task is happy with the result. That’s not very well-structured, and in fact it smells like an anti-pattern. (Like when engineers write acceptance tests. But I digress.)
So, my experiment, (which is ongoing, so I don’t have outcomes to report), is to take the concept of a user story and extend it to the various functional roles within our team. So there will be “role stories” but they can be formulated as typical user stories, and I will refer to them as user stories throughout. This concept is actually not unusual, or shouldn’t be, but I’ve only seen hints of this thinking on teams that I’ve worked with. I’m describing it here for my own benefit and because I assume many other teams out there fail to take a structured approach to self-organizing. (Certainly true of most of the teams I have worked on.)
So the first user stories from my example would belong to the build manager. As a build manager I need to remove unused dependencies from our modules… etc. By formulating the tasks in this way, we can not only improve the quality of the product (which will benefit the stakeholder), we allow team members to have input into the tasks that they are assigned. It is a mechanism to address stressors and productivity inhibitors. This improves morale, and, done correctly, improves competence.
The next story would be for the quality assurance role. It would go something like this: as the quality assurance owner, I want to improve our test coverage so that defects raised by testers are reduced by at least 50%. Acceptance criteria will be branch coverage of 80% in the following problem modules: … Of course, if the testers continue to raise defects, the issue would need to be revisited. Presumably the team would agree that this story makes sense, and would be able to break it down into small tasks, which should always be done.
The next problem, facilitating team growth, is one that has been a struggle for most teams I’ve worked on. For that reason, I tend to think that it’s a difficult problem to solve. It can often be decomposed into small tasks and the burden shared among multiple members of the team, which I see as an upside. As a team member, I want to assist new team members by reducing the time it takes them to begin being productive without impacting the delivery of the sprint backlog. I’ll know this is possible when we have an accurate deployment diagram, a project overview guide, and a workstation setup guide.
Where this thought experiment starts to get interesting is when we think about maintaining the “product” of these user stories. (Something the product owner really should care about.) So when a backlog shows a story like, as a product user I need to be able to search for widgets, we see that there are implied user stories (role stories!) that exist in parallel. That is, as an engineer I need to create a Maven project that exposes an HTTP-based search API, and also, as a build manager I need to add new projects into our continuous integration environment, and of course as a test designer, I need to write functional tests against all HTTP-based services using SoapUI. (You do test your code, right?) So the interesting property of this thought experiment is the natural decomposability of user stories into tasks, and those tasks are themselves a sort of user story (role story, in my terminology). Add stories for a technical writer, deployment manager, and production support team and you’ve avoided a lot of technical debt.
The other point that must be mentioned, thought it is obvious, is that the tasks associated with these stories may be completed by the same person. One person may perform multiple functional roles. For example, a tech lead may often function as an engineer, team growth facilitator, build manager, documentation maintainer, etc., possibly all within the same day.
Functional decomposition, which is really all I’m talking about, helps to narrow the scope of a task in order to make it manageable. Whether we’re writing use cases, user stories, software, or documentation, functional decomposition is an important tool that is often ignored yet very valuable.
Badgile
Badgile, Adj.
A condition where the product owner continually badgers an agile team for failing to deliver unspecified acceptance criteria. Typically experienced during end-of-sprint demos and retrospectives.
Unit tests as proofs
Lately I’ve found myself writing what I would consider to be odd unit tests. Odd because they are not written to test my code, or really to test any code that is used in my current project. Rather, these tests have served as proofs of the behavioral characteristics of target deployment environments.
One test I wrote was to demonstrate that a particular filesystem location was not writable from within an app server. This had been preceded by proof that assuming the user credentials used by the app server process did indeed have write access. In this situation, quite strangely, the app server had failed to acquire write permissions. Since the application failures we were seeing appeared to be due to this filesystem permissions issue, no further troubleshooting effort was required on my part because I had proven that the failure was not in the application layer.
Another test I wrote to demonstrate that certain HTTP headers were not being passed to a Java Servlet container. Additionally, the test showed that the container was not responding appropriately when those headers were subsequently enabled. Why did this test matter? By demonstrating that the application was not defective, I was able to avoid modifying application code to work around an improperly configured environment.
Finally, I wrote a test to prove that certain response characteristics of an OAuth-protected resource were incorrect. This would have been extremely difficult to accomplish solely by use of runtime tested (e.g. navigation in a browser) because the OAuth calls were not under the control of the browser. By using automated tests, I not only shortened my debugging time, I also was able to send my test case to another engineer for verification.
By writing tests at the start of problem investigations, I am able to see the impact of changes. (It is often the case that an attempt to fix a problem will create a new problem elsewhere.) I am able to repeat my test many times with minimal effort. I am also able to share my tests with others; one benefit of that is that I may make mistakes that are more easily spotted by others.
Because of all this, I have lately been thinking of unit tests more along the lines of mathematical proofs, rather than just a way to exercise a set of CPU instructions. Properly constructed, a unit test can serve as a very practical sort of mathematical proof. And in my case, these proofs helped to spare me from needless time-wasting activities. By rigorously asserting the conditions required for success or failure, I was able to repeatedly and consistently isolate the causes of these various problems I had encountered. In all of these cases, I would have saved even more time had I started my troubleshooting efforts by writing the test first. Test-driven troubleshooting isn’t always possible, but I do recommend that you consider it, especially if you value your time and do not wish to waste it on needless activities.
Windows 2003’s command interpreter and the Consolas font
Windows 2003 does not come with the Consolas font enabled for use by cmd.exe. This is of course very unfortunate, because the only 2 fonts available (a raster font and Lucida Console) are both substandard options. In my previous post I included shortcut files that assume Consolas is available. If you’re stuck with an older version of Windows, the information that follows may be helpful to you.
Copy and paste the following two lines into a command prompt. You will be logged out. When you have logged back in, your command windows will be able to use Consolas.
(reg add "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Console\TrueTypeFont" /v 00 /d Consolas
logoff)
Now you can take full advantage of the shortcuts you downloaded from my previous post.
(This information is based on a blog post by Bill Hill – http://blogs.msdn.com/b/ie/archive/2008/04/22/give-your-eyes-a-treat.aspx.)
Beefing up the standard Windows command interpreter
While out-of-the-box features for Windows’ command interpreter (cmd.exe) have mildly improved over the years (command completion turned on by default comes to mind), the defaults are still boring. Really, really boring. And kinda ugly, too.
Beyond the boring and the ugly, however, stands the dysfunctional. A tiny scroll buffer means you will quickly and easily lose data. Mouse support is turned off by default. The window is too small by default. The font could be better…
So, to spare you, the reader, the time needed to fix all those settings, I created some shortcuts (.lnk files) that will present you with reasonable defaults. You can download them all in a zipfile, or individually select the ones you’re interested in.
Here’s a screenshot with four examples:

And here are the individual links:
Of course, if you have improvements/suggestions for working with cmd.exe, I’d be very interested in hearing them.
Hudson+Maven unable to locate javac
I recently broke pretty much all of the builds on one of the Hudson servers at work. (Oops!) This server happened to have a few dozen fairly homogeneous Maven+Java builds all building with JDK 5. This is all hosted on a Tomcat instance running on a standalone JRE 6.
All I did was add a JDK. Really. Adding a second JDK to the Hudson configuration broke every single Maven+Java build. This is the (initially very confusing) error message I got:
[ERROR] Unable to locate the Javac Compiler in: C:\Java\jre6\..\lib\tools.jar
The reason I added a second JDK to Hudson is because I needed to run builds with JDK 6. This seems pretty mundane. As it turns out, Hudson gets really confused when changing from “one installed JDK” to “more than one installed JDK” and starts looking for the default JDK in JAVA_HOME (aka Java System property java.home). Naturally, this means that Hudson has no obvious way to specify a default JDK. It also means that my builds could not run, because Hudson gets its java.home value from Tomcat, which is running under a JRE, not a JDK.
This is an example of poor software design, for three reasons. First, because it leads to unexpected consequences. (All builds now default to a new, and in this case non-existent, JDK.) Second, because builds can be configured to run under a JDK called “(Default)” yet there is no obvious mechanism to actually specify what this default JDK actually is. Third, finally, and most importantly: because it leads to harmful outcomes (i.e. failed builds).
While I suspect the correct course of action would be to modify Hudson to directly support a “default JDK” feature, there is in fact a workaround. On the configuration page there is a “Global Properties” section with a checkbox entitled “Environment variables.” If you select this, you can override java.home with your own JRE, one that is part of a JDK installation. Here is an example:
C:\Java\jdk-1.5.0_22\jre
So far, this override only seems to work when pointing to a JRE that is inside a JDK. Yes, that trailing “jre” really matters. I went ahead with this setting because I didn’t want to edit all of the failed builds. I hope this isn’t a bad idea that will come back to bite me later, but I really don’t want to edit all those builds.
Good luck with all your continuous integration efforts!
ClassCastException: Given property is single cardinality
I am attempting to use a Data Handler in a WPS7 mediation to convert a string value to a business object and assign it to an XML anyType field. I need this to work because my output type can vary but is useless if it’s not de-serialized from a simple string (can’t run XPath on it, do message type routing, etc.).
org.apache.commons.jxpath.JXPathException: Exception trying to set value with xpath /context/transient/any; java.lang.ClassCastException: Given property is single cardinality
FFDC Exception:java.lang.ClassCastException SourceId:com.ibm.ws.sibx.mediation.primitives.util.xpath.DataObjectPropertyPointer.setValue ProbeId:1:659:1.33 Reporter:com.ibm.ws.wbi.xpath.impl.jxpath.DataObjectPropertyPointer@7d757d75 java.lang.ClassCastException: Given property is single cardinality at com.ibm.ws.bo.bomodel.util.BOUtil.getList(BOUtil.java:3073) at com.ibm.ws.bo.bomodel.impl.DynamicBusinessObjectImpl.getList(DynamicBusinessObjectImpl.java:742) at com.ibm.ws.wbi.xpath.impl.jxpath.DataObjectPropertyPointer.setValue(DataObjectPropertyPointer.java:595) at org.apache.commons.jxpath.ri.JXPathContextReferenceImpl.setValue(JXPathContextReferenceImpl.java:503) at org.apache.commons.jxpath.ri.JXPathContextReferenceImpl.setValue(JXPathContextReferenceImpl.java:418) at org.apache.commons.jxpath.ri.JXPathCompiledExpression.setValue(JXPathCompiledExpression.java:73) at com.ibm.ws.wbi.xpath.impl.jxpath.PrivilegedCompiledExpression$10.run(PrivilegedCompiledExpression.java:460) at java.security.AccessController.doPrivileged(AccessController.java:202) at com.ibm.ws.wbi.xpath.impl.jxpath.PrivilegedCompiledExpression.setValue_2(PrivilegedCompiledExpression.java:453) at com.ibm.ws.wbi.xpath.impl.jxpath.PrivilegedCompiledExpression.setValue(PrivilegedCompiledExpression.java:447) at com.ibm.ws.wbi.xpath.impl.jxpath.XPathExpressionImpl.setValue(XPathExpressionImpl.java:162) at com.ibm.ws.sibx.mediation.primitives.datahandler.DataHandlerMediation.processTransformation(DataHandlerMediation.java:697) at com.ibm.ws.sibx.mediation.primitives.datahandler.DataHandlerMediation.mediate(DataHandlerMediation.java:669) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.performInvocation(JavaMediationPrimitive.java:684) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.invoke(JavaMediationPrimitive.java:369) at com.ibm.ws.sibx.scax.mediation.engine.MediationPrimitive.invokeConnections(MediationPrimitive.java:354) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.fireOutputTerminals(JavaMediationPrimitive.java:783) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.performInvocation(JavaMediationPrimitive.java:706) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.invoke(JavaMediationPrimitive.java:369) at com.ibm.ws.sibx.scax.mediation.engine.MediationPrimitive.invokeConnections(MediationPrimitive.java:354) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.fireOutputTerminals(JavaMediationPrimitive.java:783) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.performInvocation(JavaMediationPrimitive.java:706) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.invoke(JavaMediationPrimitive.java:369) at com.ibm.ws.sibx.scax.mediation.engine.MediationPrimitive.invokeConnections(MediationPrimitive.java:354) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.fireOutputTerminals(JavaMediationPrimitive.java:783) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.performInvocation(JavaMediationPrimitive.java:706) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.invoke(JavaMediationPrimitive.java:369) at com.ibm.ws.sibx.scax.mediation.engine.MediationPrimitive.invokeConnections(MediationPrimitive.java:354) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.fireOutputTerminals(JavaMediationPrimitive.java:783) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.performInvocation(JavaMediationPrimitive.java:706) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.invoke(JavaMediationPrimitive.java:369) at com.ibm.ws.sibx.scax.mediation.engine.MediationPrimitive.invokeConnections(MediationPrimitive.java:354) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.fireOutputTerminals(JavaMediationPrimitive.java:783) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.performInvocation(JavaMediationPrimitive.java:706) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.invoke(JavaMediationPrimitive.java:369) at com.ibm.ws.sibx.scax.mediation.engine.MediationPrimitive.invokeConnections(MediationPrimitive.java:354) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.fireOutputTerminals(JavaMediationPrimitive.java:783) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.fireOutputTerminalOnly(JavaMediationPrimitive.java:532) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.invoke(JavaMediationPrimitive.java:407) at com.ibm.ws.sibx.scax.mediation.engine.MediationPrimitive.invokeConnections(MediationPrimitive.java:354) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.fireOutputTerminals(JavaMediationPrimitive.java:783) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.performInvocation(JavaMediationPrimitive.java:706) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.invoke(JavaMediationPrimitive.java:369) at com.ibm.ws.sibx.scax.mediation.engine.MediationPrimitive.invokeConnections(MediationPrimitive.java:354) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.fireOutputTerminals(JavaMediationPrimitive.java:783) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.performInvocation(JavaMediationPrimitive.java:706) at com.ibm.ws.sibx.scax.mediation.engine.JavaMediationPrimitive.invoke(JavaMediationPrimitive.java:369) at com.ibm.ws.sibx.scax.mediation.engine.MediationPrimitive.invokeConnections(MediationPrimitive.java:354) at com.ibm.ws.sibx.scax.mediation.engine.Input.invoke(Input.java:195) at com.ibm.ws.sibx.scax.mediation.engine.RequestFlow.invokeFlow(RequestFlow.java:135) at com.ibm.ws.sibx.scax.mediation.engine.MediationFlow.invokeRequestFlow(MediationFlow.java:157) at com.ibm.wsspi.sibx.mediation.flow.ejb.MediationFlowBean.invokeRequestFlow(MediationFlowBean.java:255) at com.ibm.ws.sibx.scax.mediation.component.SIBXMediationFlowComponentImpl.invokeRequestFlow(SIBXMediationFlowComponentImpl.java:134) at com.ibm.ws.sibx.scax.runtime.handler.MFCImplementationHandler.processMessage(MFCImplementationHandler.java:204) at com.ibm.ws.sca.internal.message.impl.MessageDispatcherImpl.processMessageWithPCI(MessageDispatcherImpl.java:755) at com.ibm.ws.sca.internal.message.impl.MessageDispatcherImpl.processMessage(MessageDispatcherImpl.java:1607) at com.ibm.ws.sca.internal.message.impl.ManagedMessageImpl.process(ManagedMessageImpl.java:980) at com.ibm.ws.sca.uow.handler.UOWNativeWASStrategyImpl.transactionGlobalActivitySessionAny(UOWNativeWASStrategyImpl.java:853) at com.ibm.ws.sca.uow.handler.JoinUOWNativeHandler.processMessage(JoinUOWNativeHandler.java:207) at com.ibm.ws.sca.internal.message.impl.MessageDispatcherImpl.processMessageWithPCI(MessageDispatcherImpl.java:766) at com.ibm.ws.sca.internal.message.impl.MessageDispatcherImpl.processMessage(MessageDispatcherImpl.java:1607) at com.ibm.ws.sca.internal.message.impl.ManagedMessageImpl.process(ManagedMessageImpl.java:980) at com.ibm.ws.sca.internal.webmodule.handler.WebModuleContextSwitchHandler.processMessage(WebModuleContextSwitchHandler.java:146) at com.ibm.ws.sca.internal.message.impl.MessageDispatcherImpl.processMessageWithPCI(MessageDispatcherImpl.java:766) at com.ibm.ws.sca.internal.message.impl.MessageDispatcherImpl.processMessage(MessageDispatcherImpl.java:1607) at com.ibm.ws.sca.internal.message.impl.ManagedMessageImpl.process(ManagedMessageImpl.java:971) at com.ibm.wbit.comptest.controller.invocation.impl.Invoker.managedInvoke(Invoker.java:316) at com.ibm.wbit.comptest.controller.invocation.impl.Invoker.invoke(Invoker.java:196) at com.ibm.wbit.comptest.controller.invocation.impl.BaseInvocationHandler.invoke(BaseInvocationHandler.java:64) at com.ibm.wbit.comptest.controller.invocation.impl.InvocationManager.doInvoke(InvocationManager.java:67) at com.ibm.wbit.comptest.controller.invocation.impl.InvocationManager$1.run(InvocationManager.java:48) at java.security.AccessController.doPrivileged(AccessController.java:224) at javax.security.auth.Subject.doAs(Subject.java:495) at com.ibm.websphere.security.auth.WSSubject.doAs(WSSubject.java:131) at com.ibm.wbit.comptest.controller.invocation.impl.InvocationManager.invoke(InvocationManager.java:42) at com.ibm.wbit.comptest.controller.command.impl.BaseCommandHandler.doCommand(BaseCommandHandler.java:62) at com.ibm.wbit.comptest.controller.command.impl.CommandProcessor.doCommand(CommandProcessor.java:70) at com.ibm.wbit.comptest.controller.command.impl.CommandProcessor.doCommand(CommandProcessor.java:42) at com.ibm.wbit.comptest.controller.servlet.TestControllerServlet.process(TestControllerServlet.java:163) at com.ibm.wbit.comptest.controller.servlet.TestControllerServlet.doPost(TestControllerServlet.java:127) at javax.servlet.http.HttpServlet.service(HttpServlet.java:738) at javax.servlet.http.HttpServlet.service(HttpServlet.java:831) at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1663) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:939) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:502) at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:181) at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:91) at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:864) at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1592) at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:186) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:452) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:511) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:305) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:276) at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214) at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113) at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165) at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217) at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161) at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138) at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204) at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775) at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905) at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1613)
If I cast the target anyType field to the appropriate schema type, this error message goes away. Unfortunately, I don’t know what the type will be ahead of time. This is obvious because I’m using anyType. I repeat: this is obvious because I’m using anyType!
scp without using scp—redux
I wrote a short post about 3 years ago called scp without using scp, and I have found it very helpful to refer back to it from time to time. Today is one of those days, but—surprisingly—it was helpful to someone other than myself. As a result, I decided to post a new example.
Let’s say we want to compress a directory, transfer it over the network, then decompress it onto our current machine, and do the whole thing without creating temporary files on either system. Here’s one way to do it:
sudo ssh -p 2222 user@host 'cd /home/user/ ; tar -c "iwant/" | bzip2 -9 -c' \ | bunzip2 -c | sudo tar -x
Here’s how it works:
- sudo invokes ssh (you may need to enter a password).
- SSH connects to the host (server) on port 2222 and authenticates. (You may need to enter a password). A shell is now open on the host.
- The shell on the host changes its working directory to /home/user.
- Tar begins to compress the files in the iwant directory and sends its output to STDOUT.
- bzip2 compresses the data from STDIN (sent by tar) and sends it to STDOUT
- The data from STDOUT (sent by bzip2) goes across the network to the machine that initiated the SSH connection and to that process’ STDOUT.
- ssh’s STDOUT is piped to bunzip2’s STDIN. bunzip2 then writes decompressed data to STDOUT.
- sudo starts up tar, which reads from STDIN (uncompressed data from bunzip2) and recreates the directory structure, file names, and file content.
To be more concise, the sequence is: connect, tar, compress, network transmit, uncompress, untar, disconnect. The first sudo call is probably unnecessary, but the second one is required if you lack write permission to your current directory.
The quickest way to test this is to ssh to localhost. Getting the command exactly right can be tricky, so if you’re trying to transfer large files this way, it’s a good idea to test the command locally first.
So, there you have it: compressed data transfers without temp files. Whether or not you get faster transfer speeds is another story. Each situation is different. You can find the answer to that on your own.
