tag:blogger.com,1999:blog-62761654471676200362024-03-05T07:14:54.592-08:00The Flying Spaniel Software Design BlogMy take on "good" software design.Morgan Conradhttp://www.blogger.com/profile/15963409751532704983noreply@blogger.comBlogger46125tag:blogger.com,1999:blog-6276165447167620036.post-80627310143278629292016-11-10T10:27:00.003-08:002016-11-10T10:30:57.616-08:00ES6 review, part-2Continuing (belatedly) from the <a href="http://flyingspaniel.blogspot.com/2016/05/es6-review-part-1.html">previous review</a> of some new language features of Javascript ES6. Again, based upon <i><a href="http://shop.oreilly.com/product/0636920033769.do">ES6 & Beyond</a></i> by Kyle Simpson <span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"> </span><span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;">(O'Reilly, (c) 2016) and using <a href="https://en.wikipedia.org/wiki/Chess_annotation_symbols">chess annotations</a> (!, ?, etc.) to rate the features.</span><br />
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><b>Concise Properties!</b></span><br />
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;">Saves typing, especially if you use long, informative variable names, which encourages their use, which is a very good thing. However, I'm a little concerned in all of these "shortcuts" about typos and or accidentally polluting the global name space.</span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><b>Concise Methods!?</b></span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;">A shorter way to create function <i>expressions.</i> The whole <i>expression </i>vs. <i>declaration </i>stuff gives me a headache, and, coming from a Java "like classes" background I tend to use the later. So not sure how much I'll use the new feature. </span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;">Also, these concise method expressions fall apart (and are tricky to debug) when you are nested deep withing other functions then calling things recursively. To which I say <b>don't do that!</b> The code for which they fail is extremely complex code which is probably best refactored.</span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><b>Getter / Setter ?</b></span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white;"><span style="font-size: 17.8px;">I'm torn on this one. Obviously, coming from a Java background, I like the concept. However, I also like explicitness, and operator overloading is confusing and generally held to be a bad thing. With the new getters/setters, when I go</span></span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "courier new" , "courier" , monospace;"><span style="background-color: white; font-size: 17.8px;"> var x = foo.bar;</span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;">I don't know if this is simple trivial property lookup (which it sure looks like!), or this is calling a function that will spend 10 minutes calculating pi, then become self-aware and launch Judgement Day. Following <a href="https://www.python.org/dev/peps/pep-0020/">Zen of Python</a>, Rule #2: "Explicit is better than implicit", I think it's preferable to show the function call:</span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "courier new" , "courier" , monospace;"><span style="background-color: white; font-size: 17.8px;"> var x = foo.getBar();</span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><b>Computed Property Names?!</b></span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;">Meh, again not so sure. If I have a lot of properties (or methods) starting with "Hillary-", and another set with "Trump-", do I want them all mixed into a single object, or isn't it simpler to subdivide and have</span></span><br />
<pre class="language-javascript"><code>
var foo = {
Hillary: {
party: "democrat",
taxes: function() { ... }
},
Trump: {
party: "yes",
taxes: function() { return undefined; }
}
}
</code></pre>
<br />
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;">and access them via</span><br />
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-size: 17.8px;"><span style="font-family: "courier new" , "courier" , monospace;"> foo[name][property]?</span></span><br />
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;">I don't see any great advantage is mixing them all together with foo[name+property]. However, they are used a lot for symbols, and can be useful for concise methods and generators, so I'll keep my mind open.</span><br />
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><b>Super!</b></span><br />
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;">hard to argue much with this.</span><br />
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><b>Template Literals (a.k.a. Interpolated String Literals)</b></span><br />
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><b><br /></b></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;">I agree with the complaint that these aren't "templates" in the sense that many use them. They aren't Mustache, JADE, whatever. However, the ability to use</span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "courier new" , "courier" , monospace;"><span style="background-color: white; font-size: 17.8px;">"Hello ${firstname} ${lastname}!"</span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;">instead of</span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "courier new" , "courier" , monospace;"><span style="background-color: white; font-size: 17.8px;">"Hello " + firstname + " " + lastname + "!";</span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white;"><span style="font-size: 17.8px;">strikes me as a very very good thing. One can also put functions inside the braces, which, if used with discretion, is also great.</span></span></span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="color: #333333; font-family: "courier new" , "courier" , monospace;"><span style="background-color: white;"><span style="font-size: 17.8px;">"Hello ${decodeURI(nameFromAForm)}"</span></span></span><br />
<span style="color: #333333; font-family: "courier new" , "courier" , monospace; font-size: x-small;"><span style="background-color: white;"><br /></span></span>
<span style="font-family: "georgia" , "times new roman" , serif; font-size: x-small;"><b>Arrow Functions</b></span><br />
<span style="font-family: "georgia" , "times new roman" , serif; font-size: x-small;"><br /></span>
<span style="font-family: "georgia" , "times new roman" , serif; font-size: x-small;">I'll talk more later about these. He does point out a lot of "gotchas" and limits the cases where they should be used.</span><br />
<span style="font-family: "georgia" , "times new roman" , serif; font-size: x-small;"><br /></span>
<span style="font-family: "georgia" , "times new roman" , serif; font-size: x-small;"><b>for..of Loops!!</b></span><br />
<span style="font-family: "georgia" , "times new roman" , serif; font-size: x-small;"><br /></span>
<span style="font-family: "georgia" , "times new roman" , serif; font-size: x-small;">Why did it take a gazillion years to add this obviously great feature? Yes, I guess the functional guys will say you could use </span><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">forEach</span><span style="font-family: "georgia" , "times new roman" , serif; font-size: x-small;">, but this is simpler for most.</span><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white;"><span style="font-size: 17.8px;"><br /></span></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white;"><span style="font-size: 17.8px;"><br /></span></span></span>
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white;"><span style="font-size: 17.8px;"><br /></span></span></span>
<b>Notes</b><br />
<span style="color: #333333; font-family: "georgia" , serif;"><span style="background-color: white; font-size: 17.8px;"><br /></span></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;">Astute readers will note that I'm not using <a href="http://prismjs.com/">prism.js </a>for syntax highlighting. Seems to work well. However, it's tedious in Blogger to go to HTML edit mode, and when I'm there I see that the HTML code is a disaster of crapola so it's hard to find the code you just types to surround it by the</span><span style="background-color: white; color: #333333; font-family: courier new, courier, monospace; font-size: 17.8px;"> pre and code </span><span style="background-color: white; color: #333333; font-family: georgia, serif; font-size: 17.8px;">tags. I'm going to start playing with </span><a href="https://ghost.org/" style="font-family: georgia, serif; font-size: 17.8px;">Ghost </a><span style="background-color: white; color: #333333; font-family: georgia, serif; font-size: 17.8px;">and this blog is likely to migrate there soon. And Blogger constantly changing my font size is just so annoying.</span><br />
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<span style="background-color: white; color: #333333; font-family: "georgia" , serif; font-size: 17.8px;"><br /></span>
<br />
<br />Morgan Conradhttp://www.blogger.com/profile/15963409751532704983noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-70928768980326374022016-05-06T16:51:00.005-07:002016-05-06T16:51:53.792-07:00ES6 Review, part-1This is a first post reviewing the new features of JavaScript ES6. It is also a short review of <i><a href="http://shop.oreilly.com/product/0636920033769.do">ES6 & Beyond</a></i>, by Kyle Simpson (O'Reilly, (c) 2016), which I'm using to learn most of the new stuff. <a href="https://github.com/getify/You-Dont-Know-JS/tree/master/es6%20%26%20beyond">Also on GitHub</a>. I will rate things using <a href="https://en.wikipedia.org/wiki/Chess_annotation_symbols">chess annotations</a>, where an exclamation mark is "good" and a question mark is "dubious".<br />
<br />
As for my background and biases, for 20+ years I was a traditional desktop app programmer, mainly in C and Java. I'm fairly new (~4 years) to JavaScript. I often appreciate, but sometimes hate, the differences between it and Java. I've used <a href="https://nodejs.org/en/">NodeJS </a>and some of its most common modules (<a href="http://expressjs.com/">Express</a>, <a href="https://www.npmjs.com/package/request">Request</a>), but otherwise, have used few of the mainstream JS libraries such as <a href="https://jquery.com/">JQuery </a>and <a href="https://angularjs.org/">Angular</a>.<br />
<br />
<br />
<b>Intro</b><br />
<br />
The book is moderate sized (~260 pages, trade paperback sized (6" x 9")) and meets O'Reilly's typical high standards for fonts, printing, binding etc. It's a little disappointing that they ran out of the traditional animal pictures to put on the covers. :-)<br />
<br />
<br />
Let's start with Chapter 2, which covers new Syntax. There's a lot. <br />
<br />
<b>Block Scoping!!</b><br />
<br />
Welcome to the <i>20th</i> century JavaScript. Anything that reduces the need for an <a href="https://en.wikipedia.org/wiki/Immediately-invoked_function_expression">IIFE hack</a> is a plus in my book.<br />
<br />
<b>Let!!</b><br />
<br />
Instead of <span style="font-family: "courier new" , "courier" , monospace;">var x,</span> which is subject to the occasional bugs of <a href="http://www.w3schools.com/js/js_hoisting.asp">hoisting</a>, or polluting namespaces, using <span style="font-family: "courier new" , "courier" , monospace;">let x</span> declares the variable with block scope <i>from that point forward</i>, much like C and Java. Using that variable before it's declaration results in a ReferenceError from a "Temporal Dead Zone" (TDZ). This sounds confusing and the terminology is off-putting, but the simple answer is to <i>declare your variables before using them!</i> Just like almost every other language.<br />
<br />
Unfortunately, the author goes off on a multi-page tangent describing his preferred version for <span style="font-family: Courier New, Courier, monospace;">let </span>syntax, which is less conventional and differs from C/Java tradition. Since this alternate syntax was <i>not adopted by ES6</i>, the discussion is confusing and useless. In general, I like the opinions expressed by Mr. Simpson in the book, but here was one case where he went off base.<br />
<br />
<b>Let with a For Statement!</b><br />
<br />
A minor benefit of <span style="font-family: "courier new" , "courier" , monospace;">let </span>over <span style="font-family: "courier new" , "courier" , monospace;">var </span>in a loop is that <span style="font-family: "courier new" , "courier" , monospace;">let </span>redeclares the variable <i>each iteration</i>, so it is "sticky" if used in a callback. Mr. Simpson provides a precise and clear example of this benefit.<br />
<br />
<b>const!</b><br />
<br />
Immutability has many advantages, and in ES6 declaring a variable <span style="font-family: "courier new" , "courier" , monospace;">const </span>makes this clearer. Note that in the case of a reference to an object, say an array, only the <i>reference </i>is immutable, not the <i>contents of the array</i>. That limitation is unfortunate but the same as many other languages. Mr. Simpson provides a concise and clear example.<br />
<br />
<b>Block Scoped Functions</b><br />
<br />
Meh, o.k. If you write a lot of complex recursive stuff it's useful, but I don't see a huge need for this in typical code, and there are some incompatibility issues.<br />
<br />
<b>Spread! Rest (a.k.a. "Gather")!!</b><br />
<br />
Both of these use "..." in your code, which might be slightly confusing at first, but I think in practice you'll get used to it, and it mimics existing C and Java syntax, so it was a good choice.<br />
<br />
When ... is used before an iterable (see later), such as an array, it "spreads out" the individual elements of that iterable. Mr. Simpson provides nice examples of how this can be used to conveniently replace <span style="font-family: "courier new" , "courier" , monospace;">apply()</span> or <span style="font-family: "courier new" , "courier" , monospace;">concat().</span><br />
<br />
In alternative places, ... <i>gathers </i>a group of variable into an array. The book calls this "rest" because it is gathering "the rest of the arguments". I'd much prefer "gather" or "<a href="https://www.eskimo.com/~scs/cclass/int/sx11b.html">varargs</a>". Since a great place to use this syntax is to gather any variable arguments passed to a function at the end of the argument list. e.g.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">function foo(a, b, ...varargs) {}</span><br />
<br />
This mimics behavior in other languages such as <a href="https://docs.python.org/2/tutorial/controlflow.html#arbitrary-argument-lists">Python</a> and <a href="http://docs.oracle.com/javase/1.5.0/docs/guide/language/varargs.html">Java</a>. Note that a common use case will be to rewrite code that used the deprecated "array-like" <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/arguments">arguments array</a>. Instead of chanting the mantra "Array.prototype.slice.call":<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">function foo() {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> var args = Array.prototype.slice.call(arguments);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> // now args is a real array</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<br />
Just do<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">function foo(...varargs) {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<br />
<b>Note</b>: This is not explicitly mentioned in the book, but if there are no "rest of" arguments to gather, varargs will be set to an empty array, not undefined. IMO this is correct behavior.<br />
<br />
<br />
<b>Default Parameter Values!</b><br />
<br />
These are superior to streams of<br />
<span style="font-family: Courier New, Courier, monospace;">x = x || 6;</span><br />
in your code. <br />
<br />
You can also call functions or invoke an IIFE in your default, but, IMO, this is probably getting too complex or cute. The book hints at this and details several "gotchas". But one nice use-case is to provide a default "do nothing" callback function in your function declaration, e.g.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">function asyncFoo(url, callback=function(){}) {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<br />
<br />
<b>Destructuring ?!</b><br />
<br />
Have to say, I don't see the point in this, the book doesn't provide any "killer use cases", and it's confusing. If anybody does have a killer use case, please let me know. Until then, count me a skeptic.<br />
<br />
Let's say a function returns multiple results, e.g. in an array or an object.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">function get3DCoordinates() { </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> return [1,2,3]; </span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">function getLocation() {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> return {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> lat : 1;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> lon : 2;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<br />
<br />
Before ES6, you'd go<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">var c3 = get3DCoordinates();</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">// use c3[0], c3[1], c3[2]as needed...</span><br />
<br />
or<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">var loc = getLocation();</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">// use loc.lat, loc.lon as needed...</span><br />
<br />
<br />
With ES6, you can go<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">var [x,y,z] = get3DCoordinates();</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">// use x, y, z instead...</span><br />
<br />
or<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">var { lat: lat, lon:lon } = getLocation();</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">// use lat, lon instead...</span><br />
<br />
<br />
The syntax with the variables on the left is a bit confusing, but I could get used to it. Fundamentally, I question this usefulness of this. When a function returns multiple values, those values usually <i>belong together</i>. If they don't belong together, why is a single function returning them? You are likely violating the <a href="https://www.eskimo.com/~scs/cclass/int/sx11b.html">Single Responsibility Principle</a>. <br />
<br />
<i>What's the advantage of splitting apart things that belong together?</i><br />
<br />
An another counterexample, what if you needed the location of two things? What looks like better code to you?<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">var sanFran = getLocation(94101);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">var seattle = getLocation(98101);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">var dLat = sanFran.lat - seattle.lat;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">var dLon = </span><span style="font-family: "courier new" , "courier" , monospace;">sanFran.lon - seattle.lon;</span><br />
<br />
or<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">var { latSF, lonSF } = getLocation(94101);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">var { latSeattle, lonSeattle} = getLocation(98101);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">var dLat = larSF - latSeattle;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">var dLon = lonSF - lonSeattle;</span><br />
<br />
I prefer the old fashioned approach, but maybe your mind works differently. What if you need the location of N things?<br />
<br />
Pages 26-38 of the book cover many more complex cases of destructuring. With, IMO, no killer use case. For example, is<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">var { model : { User } } = App;</span><br />
<br />
really any improvement over<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">var User = App.model.User;</span><br />
<br />
<i>No. </i> The new syntax is confusing, it's more typing (in this case), and, most importantly, <i>it's very complex syntax in the typing</i>. Instead of a couple of dots, flowing left to right, in standard western 1,2,3 order, as we are all used to, you have to properly nest brackets, and the order of the fields is all mixed up - not even reversed, the order is 2,3,1! The programmer has to <i>think</i>, usually <a href="https://en.wikipedia.org/wiki/Don%27t_Make_Me_Think">a bad thing</a>.<br />
<br />
Until I see a good use case for destructuring, it seems like a confusing addition to the language with little usefulness. Mr. Simpson provides an example where you can combine preferences/settings from two different objects,<i> e.g.</i> user-preferences and default values. However, the example only works if you know ahead of time (and put in the code) <i>all </i>of the possible fields. In my experience, preferences expand over time and come from different projects, so this would be difficult or impossible to maintain. Without a good use case, I don't see much value in destructuring. What do you think about it?<br />
<br />
Next time, we will continue with even more syntax changes, starting with Object Literal Expressions. See you then.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-57413437066886934902016-03-24T15:19:00.001-07:002016-03-24T15:19:54.522-07:00Learning Google MapsInspired by the upcoming <a href="https://r2ak.com/">Race to Alaska</a>, I put together a <a href="http://www.nextq.info/R2AK.html">web page</a> that integrates the nice JSON weather reports from the <a href="http://openweathermap.org/current">OpenWeatherMap API</a>, plus some XML format reports from <a href="http://www.ndbc.noaa.gov/get_observation_as_xml.php?station=46131">NOAA buoys</a>. These get drawn on a map with <a href="http://ww2010.atmos.uiuc.edu/(Gh)/guides/maps/sfcobs/wnd.rxml">wind barbs</a>. You can scroll around to see various areas on the map, and click on a station to get more details.<br />
<br />
It tracks the recent reports and attempts to compute an hourly rate of change for some statistics. Depending on the time between reports this can be an inexact science. But it helps to see if wind, pressure or temperature is changing rapidly.<br />
<br />
The back end code is written in JavaScript using <a href="http://node.js/">Node.js</a>. The front end is HTML and JavaScript and the <a href="https://developers.google.com/maps/documentation/javascript/">Google Maps Javascript API</a>.Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-79312665543377801862014-12-29T12:12:00.003-08:002014-12-29T12:16:27.897-08:00Do Frameworks get in the way? A tale of Python and PayPal IPN.I was writing some basic Python3 CGI code to handle PayPal IPN posts. <span style="background-color: white; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 13.63636302948px; line-height: 17.8048000335693px;"> </span><a href="https://developer.paypal.com/docs/classic/ipn/integration-guide/IPNIntro/" rel="nofollow" style="background: rgb(255, 255, 255); border: 0px; color: #4a6b82; cursor: pointer; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 13.63636302948px; line-height: 17.8048000335693px; margin: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">PayPal docs here</a>. <span style="background-color: white; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 13.63636302948px; line-height: 17.8048000335693px;"> </span>The IPN message authentication protocol has PayPal first POST a message to your URL. After sending back a quickie 200 OK in response to the POST, you then POST back the "complete, unaltered message", with "the same fields (in the same order) as the original message". I'm not sure this really matters, as I have seen online code that doesn't seem to worry about ordering, that claims to work. But, just to be robust, I wanted to follow the "correct" protocol.<br />
<br />
If PayPal were sending a GET, one could use <span style="font-family: Courier New, Courier, monospace;">os.environ.get('QUERY_STRING')</span>. But, for a POST, that returns <span style="font-family: Courier New, Courier, monospace;">None</span>. The <a href="https://docs.python.org/3.3/library/cgi.html">Python cgi library</a> provides a nice, standard, "handles lots of tricky cases" mechanism to read the POST fields, using <span style="font-family: Courier New, Courier, monospace;">cgi.FieldStorage()</span>. However that returns a <i>non-sorted</i> dictionary, where the order is not preserved. I reported this on a <a href="http://stackoverflow.com/questions/27556949/in-a-python3-cgi-script-how-can-i-read-the-raw-form-data-in-a-post">Stack Overflow question</a>, and asked how one could get the data exactly as sent. I mean, it was in a big String coming over the wire, <i>e.g.</i> "<span style="font-family: Courier New, Courier, monospace;">foo=bar&count=3</span>", right? This should be simple. HTTP can be complex, but this part isn't very tricky.<br />
<br />
To my surprise, nobody answered, and not many people even viewed the question. Maybe it was poorly worded. I think the real reason might be that programmers are too used to using a library or framework, such as cgi.FieldStorage. or <a href="https://www.djangoproject.com/">Django</a>, and don't understand what's actually going on deep underneath. Not picking on Python programmers here, I think the same is true in most languages.<br />
<br />
After some playing around, the answer is <b>astoundingly simple</b>. The POST data is coming over the wire as a String, so just <b>read it from <span style="font-family: Courier New, Courier, monospace;">stdin</span>.</b><br />
<br />
<span style="font-family: Courier New, Courier, monospace;">query_string = sys.stdin.read()</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
To POST everything back to PayPal, use this simple code. (Should I worry more about encoding?)<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">formData = "cmd=_notify-validate&" + query_string</span><br />
<span style="font-family: Courier New, Courier, monospace;">req = urllib.request.Request(PAYPAL_URL, formData.encode())</span><br />
<span style="font-family: Courier New, Courier, monospace;">req.add_header("Content-type", "application/x-www-form-urlencoded")</span><br />
<span style="font-family: Courier New, Courier, monospace;">response = urllib.request.urlopen(req)</span><br />
<span style="font-family: Courier New, Courier, monospace;">status = str(response.read())</span><br />
<span style="font-family: Courier New, Courier, monospace;">if (not status == "b'VERIFIED'"):</span><br />
<span style="font-family: Courier New, Courier, monospace;"> #complain/abort/whatever</span><br />
<span style="font-family: Courier New, Courier, monospace;">else:</span><br />
<span style="font-family: Courier New, Courier, monospace;"> #continue processing</span><br />
<br />
There's one drawback: now you can't get <span style="font-family: Courier New, Courier, monospace;">cgi.FieldStorage()</span> to work. When it goes to read from <span style="font-family: Courier New, Courier, monospace;">stdin</span>, there's nothing left, so it returns an empty dictionary. (the Python <a href="https://hg.python.org/cpython/file/3.3/Lib/cgi.py">cgi source code is here</a>) . So, it you also want the convenience of a dict for other purposes, such as checking on various IDs or the price they paid, you need to create your own dict. But that is also trivial:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">multiform = urllib.parse.parse_qs(</span><span style="font-family: 'Courier New', Courier, monospace;">query_string</span><span style="font-family: Courier New, Courier, monospace;">)</span><br />
<br />
Just like<span style="font-family: Courier New, Courier, monospace;"> cgi.FieldStorage()</span>, this returns a dictionary where the values are <i>lists </i>of Strings, since it is <i>possible </i>for a key to be repeated in a query, e.g. <span style="font-family: Courier New, Courier, monospace;">foo=bar&foo=car</span>. However, in practice, this is rare, and doesn't apply for the PayPal case. I guess you could always ask for the 0th item in the list - FieldStorage has some special methods for this. To simplify things, I created a nice, simple, <i>single-valued</i> form with Strings for keys and values:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">form = {}</span><br />
<span style="font-family: Courier New, Courier, monospace;">for key in multiform.keys():</span><br />
<span style="font-family: Courier New, Courier, monospace;"> form[key] = multiform.get(key)[0]</span><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com3tag:blogger.com,1999:blog-6276165447167620036.post-1369811859093644022014-11-12T15:01:00.000-08:002016-11-10T09:18:47.010-08:00Into the Clouds, deploying node.js with Modulus and OpenShiftMy Agility website, <a href="http://www.nextq.info/">www.nextq.info</a>, is up and running on <a href="http://modulus.io/">modulus.io</a>. I like Modulus. It's easy to use, has been reliable, and you don't need to do a ton of heavy-duty Unix-ese command line stuff. Their web interface does most of the work, and a simple command <span style="font-family: Courier New, Courier, monospace;">modulus deploy</span> will update your codebase. The main drawback is that they charge a small fee, $15 a month. I haven't tried any scaling yet.<br />
<br />
So lately I've also been playing with <a href="https://www.openshift.com/">OpenShift</a>. It's free for small projects, and that even includes a little scaling. It's definitely harder, more technical, and more "UNixy" than Modulus. You deploy using <span style="font-family: Courier New, Courier, monospace;">git</span>, and many commands must be done from the command line, not the web UI. They have a free book to get you started, <i><a href="https://www.openshift.com/promotions/ebook">Getting Started with Openshift</a></i>. After some fiddling, I got things going.<br />
<br />
One major issue is that Modulus and OpenShift use different environment variables for important settings like the port and ip address. So, if you want code portable across both, you will need something like this in your node code:<br />
<br />
<pre class="language-javascript"><code>
function setupConfig(config) {
if (process.env.OPENSHIFT_APP_DNS) {
config.port = process.env.OPENSHIFT_NODEJS_PORT;
config.ipAddress = process.env.OPENSHIFT_NODEJS_IP || '127.0.0.1';
config.mongoURI = process.env.OPENSHIFT_MONGODB_DB_URL;
config.isOpenshift = process.env.OPENSHIFT_APP_DNS;
}
else if (process.env.MODULUS_IAAS) { // modulus
config.port = process.env.PORT;
config.ipAddress = '0.0.0.0'; // modulus doesn't need an ip
config.mongoURI = process.env.MONGO_URI;
config.isModulus = process.env.MODULUS_IAAS;
}
// possibly more here...
return config;
}
</code></pre>
<br />
And use these values when you create the server, i.e.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">app.listen(config.port, config.ipAddress, <b>function</b>(){</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ...</span><br />
<span style="font-family: Courier New, Courier, monospace;">});</span><br />
<br />
<br />
I have the "isXXX" fields so that you can setup specific options like shutdown hooks.<br />
<br />
For OpenShift you <b>must</b> change the package.json file to point to your main class. OpenShift defaults to server.js, where most people use app.js. Be sure to have the following lines in package.json with the correct name of your main file.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">"scripts": {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> "start": "node <b>app.js</b>"</span><br />
<span style="font-family: Courier New, Courier, monospace;"> },</span><br />
<span style="font-family: Courier New, Courier, monospace;">"main": "<b>app.js</b>",</span><br />
<br />
Finally, on a scaled platform, OpenShift (using the <span style="background-color: white; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 13.63636302948px; line-height: 17.8048000335693px;"><a href="http://www.haproxy.org/">haproxy load balancer</a>) </span>"pings" your app every two seconds, quickly filling up the log file with confusing junk. There are even three (duplicate) bugs for this: <a href="https://bugzilla.redhat.com/show_bug.cgi?id=918783">918783</a>, <a href="https://bugzilla.redhat.com/show_bug.cgi?id=923141">923141</a> and <a href="https://bugzilla.redhat.com/show_bug.cgi?id=876473">876473</a>. Their suggested "fix" is to run a cron job calling <span style="background-color: white; color: #545454; font-family: 'Open Sans', Helvetica, Arial, sans-serif; font-size: 14px; line-height: 22.3999996185303px;">rhc app-tidy </span>once in a while to clear out your logs. This fixes the "too much space" issue, but you still have a big problem using the log file, cause all this pings make it harder to see any real problems. If you are brave, you could edit the haproxy.cfg file as hinted at (but not fully explained) in <a href="http://stackoverflow.com/questions/22247055/nodejs-application-on-openshift-gets-flooded-with-empty-requests">this StackOverflow post</a>. I chose an alternative.<br />
<br />
My fix is to use <a href="http://expressjs.com/">Express</a> to insert some middleware <b>before</b> the logger. The "pings" can be recognized since they have no <span style="font-family: Courier New, Courier, monospace;">x-forwarded-for</span> header. Real requests should have that field, and that's also the value you want in the logfile. At least, that works for me.<br />
<br />
First, a function to ignore these pings and not call next(). Ever the fiddler, it is wrapped in another function so that it can still show a subset of the pings - you might want to see the pings every hour or so.<br />
<br />
<pre class="language-javascript"><code>
function ignoreHeartbeat(except) {
except = except || 0;
var count = 1;
return function(req, res, next) {
if (req.headers["x-forwarded-for"])
return next(); // normal processing
if (except > 0) {
if (--count <= 0) {
count = except;
return next();
}
}
res.end();
}
}
</pre></code>
Then, in your app setup code, add this <b>before </b>you add the logger. <i>e.g.</i> (Express 3 shown)<br />
<br />
<pre class="language-javascript"><code>
app.use(ignoreHeartbeat(1800)); // 1800 is once an hour
...
app.use(express.logger(myFormat));
</pre></code>
<br />
Here's is example log data, where the ignoreHeartbeat was set to 10, so the pings should appear roughly every 20 seconds. Note how the pings have no ip address.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">Wed, 12 Nov 2014 22:08:36 GMT - - GET / 200 - 2 ms</span><br />
<span style="font-family: Courier New, Courier, monospace;">Wed, 12 Nov 2014 22:08:56 GMT - - GET / 200 - 2 ms</span><br />
<span style="font-family: Courier New, Courier, monospace;">Wed, 12 Nov 2014 22:08:59 GMT 50.174.189.32 - GET / 200 - 10 ms</span><br />
<span style="font-family: Courier New, Courier, monospace;">Wed, 12 Nov 2014 22:08:59 GMT 50.174.189.32 - GET /javascripts/jquery-jvectormap</span><span style="font-family: 'Courier New', Courier, monospace;">-1.2.2.css 200 - 17 ms</span><br />
<span style="font-family: Courier New, Courier, monospace;"> (more "real" GETs here...)</span><br />
<span style="font-family: Courier New, Courier, monospace;">Wed, 12 Nov 2014 22:09:17 GMT - - GET / 200 - 3 ms</span><br />
<span style="font-family: Courier New, Courier, monospace;">Wed, 12 Nov 2014 22:09:37 GMT - - GET / 200 - 1 ms</span>Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-88651168409297849382014-10-20T16:19:00.001-07:002014-10-20T16:19:51.319-07:00Web Scraping with node.js and CheerioI recently gave a talk at the <a href="http://www.meetup.com/BayNode/events/208388122/">BayNode Meetup</a>, about my experiences web scraping for dog agility trials using node.js and the <a href="https://www.npmjs.org/package/cheerio">cheerio module</a>. The results are used for my website, <a href="http://www.nextq.info/">www.nextq.info</a>.<br />
<br />
You can find the slides as Google Docs here: <a href="https://docs.google.com/presentation/d/1oHQx279Ot3BYA4VbWyeAQFTDWNqj61uayp6F4zPMriM/edit#slide=id.p">Web scraping with cheerio</a>. Enjoy!<br />
<br />
<br />Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-67223480595049577422014-07-23T13:03:00.000-07:002016-11-10T09:22:43.802-08:00Groovy-Like XML for Java. Simple and Sane.Parsing and navigating through XML in Java is a pain. The <span style="font-family: Trebuchet MS, sans-serif;">org.w3c.dom.*</span> classes are numerous, messy, and "old style", with no Collections, no Generics, no varargs. XPath helps a lot with the navigation part, but is still a bit complex and messy.<br />
<br />
Groovy, with XMLParser and <a href="http://groovy.codehaus.org/api/groovy/util/XmlSlurper.html">XMLSlurper</a> and their associated classes, makes this amazingly, dramatically easier. Simple and Sane. For example, <a href="http://www.manning.com/kousen/">Making Java Groovy</a> Chapter 2 has an example to parse the Google geocoder XML data to retrieve latitude and longitude. Below is the essentials of the code. The full code, which is not much longer, is <a href="https://github.com/kousen/Making-Java-Groovy/blob/6671f959c7ea9fc5e2522b4f85da1413ede71f20/ch02/groovybaseball/src/main/groovy/service/Geocoder.groovy">on GitHub here</a>.<br />
<pre class="language-java"><code>
String url = 'http://maps.google.com/maps/api/geocode/xml?' + somemore...
def response = new XmlSlurper().parse(url)
stadium.latitude = response.result[0].geometry.location.lat.toDouble()
stadium.longitude = response.result[0].geometry.location.lng.toDouble()
</pre></code>
The parsing is trivial, and navigating to the data (location.lat or location.lng) is also simple, following the familiar dot notation.<br />
<br />
Can you do something anything like this in pure Java? Not quite. So I wrote a small library, <a href="https://github.com/MorganConrad/xen">xen</a>, to mimic much of how Groovy does things. The <a href="https://github.com/MorganConrad/xen/blob/master/test/GeocoderDemo.java">full Geocoder.java code is here</a>, snippet below:<br />
<br />
<pre class="language-java"><code>
String url = BASE + URLEncoder.encode(address);
Xen response = new XenParser().parse(url);
Option 1: XPath slash style, 1 based indices
latLng[0] = response.toDouble("result[1]/geometry/location/lat");
latLng[1] = response.one("result[1]/geometry/location/lng").toDouble();
Option 2: Groovy dot style, 0 based indices
latLng[0] = response.toDouble(".result[0].geometry.location.lat");
latLng[1] = response.one(".result[0].geometry.location.lng").toDouble();
</pre></code>
Pretty close, eh?<br />
<br />
The main difference is that we can't use the dot notation directly from an object, but we can use a very similar slash notation based upon <a href="http://www.w3schools.com/xpath/xpath_syntax.asp">XPath syntax</a>. If you use XPath notation, one <b>major </b>difference from Groovy is that <i>array indices in W3C XPath are 1-based, not 0-based</i>. Therefore note that we access the 1st element of result, not the 0th. However, if the "path" starts with a . and a letter, as in the final example, the path is treated as a Groovy / "dot notation" style, with 0-based indices.<br />
<br />
So, if you want to greatly simplify parsing and navigating through XML, and/or you love how Groovy does things, please check out my (very beta!) <a href="https://github.com/MorganConrad/xen">xen library</a> which allows you to do it in Java. Currently it is compiled vs. Java 6 but I think it should be fine in Java 5. So if you need to support some Android device, or can't or don't want to integrate Groovy into your Java projects, this could be very useful.<br />
<br />
<a href="https://github.com/MorganConrad/xen">Xen library</a><br />
<a href="http://morganconrad.github.io/xen/javadocs/">JavaDocs</a><br />
<a href="https://github.com/MorganConrad/xen/blob/master/README.md">README</a><br />
<br />
The README discusses various design decisions, particularly, how my design converged upon many aspects of the Groovy design. More discussion will appear in later posts. And, be warned, this is still a very early version, 0.0.2, so there are probably bugs, some mistakes, and upcoming API changes.<br />
<br />
<br />Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-83728055012046801172014-07-23T12:22:00.000-07:002014-07-23T12:23:25.353-07:00Node for Java ProgrammersAt a recent <a href="http://www.meetup.com/BayNode/">BayNode Meetup</a>, I gave a 15 minute presentation on "Node for Java Programmers". Mainly notes on common things I did wrong coming from the Java world, and ideas or idioms to deal with them.<br />
<br />
I got some good feedback and positive responses, and recently edited the presentation.<br />
<br />
<a href="https://docs.google.com/presentation/d/1Cw_Hd3kqSKjP5QBnQRuqGipEhJJ-4XO4fh003Bo8goM/edit?usp=sharing">Here is a link to it.</a> (on Google Docs).Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-1003965499633965982014-06-12T11:03:00.000-07:002014-06-13T07:29:04.765-07:00Coding by Convention is Great... except when it isn't.<br />
<br />
"Coding by Convention" (<i>a.k.a.</i> "<a href="http://en.wikipedia.org/wiki/Convention_over_configuration">Convention over Configuration</a>") attempts to simplify programming by telling the programmer the preferred way to name or organize things. It often saves a lot of time and hassle. Without it, you write extra configuration files, typically in XML. Spring and J2EE used to require way too much configuration, with lots of stupid redundancies, something like "When I say <span style="font-family: Courier New, Courier, monospace;">Foo </span>bean I mean you to use a <span style="font-family: Courier New, Courier, monospace;">Foo.class</span>, when they go to <span style="font-family: Courier New, Courier, monospace;">myCompany.com</span>/<span style="font-family: Courier New, Courier, monospace;">order/books</span> use the <span style="font-family: Courier New, Courier, monospace;">com.myCompany.order.books</span> servlet". On the other hand there can be too much convention - I've never used <a href="http://maven.apache.org/">Maven</a>, but hear that it is particularly dictatorial and hard to modify.<br />
<br />
I'm developing a lot in JavaScript / node.js lately, and wanted the ability to save my data as either <a href="http://en.wikipedia.org/wiki/Comma-separated_values">csv files</a> or <a href="http://en.wikipedia.org/wiki/ICalendar">iCal (.ics) files</a>. Searching the <a href="https://www.npmjs.org/">NPM registry</a> finds several candidates.<br />
<br />
In csv files, the header line contains the name for each column. If using convention, this would be taken from the property key. And the property would map directly to the data in the following lines. Sometimes you would want to change this. Can you, or is it all done by convention? As I understand their documentation:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"><a href="https://www.npmjs.org/package/to-csv">to-csv</a> convention only</span><br />
<span style="font-family: Courier New, Courier, monospace;"><a href="https://www.npmjs.org/package/fast-csv">fast-csv</a> allows for transformation, but over an <i>entire row</i></span><br />
<span style="font-family: Courier New, Courier, monospace;"><a href="https://www.npmjs.org/package/json-2-csv">json-2-csv</a> convention only</span><br />
<span style="font-family: Courier New, Courier, monospace;"><a href="https://www.npmjs.org/package/json-csv">json-csv</a> allows for flexible transformations</span><br />
<br />
I ended up using json-csv, though one drawback of it's power is that it takes more work to use.<br />
<br />
On the iCalendar side, the question is how to setup or create the complex <a href="http://www.kanzaki.com/docs/ical/vevent.html">VEVENT information</a>. One could use properties named <span style="background-color: #fdfdf6; font-size: 16px;">DTSTART, </span><span style="background-color: #fdfdf6;">UID, etc. But it's <i>extremely unlikely</i> that your object has properties with those unusual and capitalized names, and the correct values. Plus, DTSTART has a complex format with a possible "DATE-TIME" option and a time format of </span><span style="background-color: #fdfdf6;">YYYYMMDDTHHMMSSZ.</span><br />
<span style="background-color: #fdfdf6;"><br /></span>
<span style="background-color: #fdfdf6;"><span style="font-family: Courier New, Courier, monospace;"><a href="https://www.npmjs.org/package/cozy-ical">cozy-iCal</a> builds VEvent objects </span></span><span style="background-color: #fdfdf6; font-family: 'Courier New', Courier, monospace;">programmatically</span><br />
<span style="background-color: #fdfdf6;"><span style="font-family: Courier New, Courier, monospace;"><a href="https://www.npmjs.org/package/ical-generator">ical-generator</a> convention (uses .start .end for DTSTART DTEND)</span></span><br />
<span style="background-color: #fdfdf6;"><span style="font-family: Courier New, Courier, monospace;"><a href="https://www.npmjs.org/package/icalevent">icalevent</a> convention (also uses .start, .end etc)</span></span><br />
<span style="background-color: #fdfdf6;"><span style="font-family: Courier New, Courier, monospace;"><a href="https://www.npmjs.org/package/icsjs">icsjs</a> builds programmatically</span></span><br />
<span style="background-color: #fdfdf6;"><span style="font-family: Courier New, Courier, monospace;"><a href="https://www.npmjs.org/package/icalendar">icalendar</a> builds VEvent objects programmatically</span></span><br />
<span style="background-color: #fdfdf6;"><span style="font-family: Courier New, Courier, monospace;"><br /></span></span>
So, it's a mishmash. And convention, while simple and convenient, doesn't always do the trick. For example, in my CSV data I'd like to include the distance from the user's location. This is obviously not even a field in the data, since it is calculated on the fly per user from the respective latitudes and longitudes. The CSV should not include the latitude and longitude - not very useful to the end user. My start and end dates are also not fields, they are stored in an array. So I definitely can't use convention. <br />
<br />
<i>Unless...</i><br />
<br />
The obvious work-around is to create a new temporary object, that meets the required convention, from the fields and data in the original, "real" object. In many cases, you would just wrote custom code to do this, especially if speed is a concern. There are also <a href="https://www.npmjs.org/package/transform-object">some modules</a> that <a href="https://www.npmjs.org/package/object-adapter">vaguely do this</a>. (Did I miss some???) But they are pretty limited. For example object-adapter can only copy values from a source object (renaming the fields), not apply any functions.<br />
<br />
So, I wrote my own general-purpose module, <a href="https://www.npmjs.org/package/remodeler">remodeler</a>. For convenience there are <span style="font-family: Courier New, Courier, monospace;">copyKeys()</span> and <span style="font-family: Courier New, Courier, monospace;">excludeKeys()</span> methods for properties you simply want to copy as-is or ignore. For the fancy stuff, you provide key / value pairs. The key is the new property name, and the value is a "transformation". <br />
<br />
If the transformation is a String, it means to copy the value from the old object, using the string as property name. For example, <span style="font-family: Courier New, Courier, monospace;">"UID", "uid"</span> would mean to create a UID property by copying the previous uid property.<br />
<br />
If the transformation is a function, it will be called via <span style="font-family: Courier New, Courier, monospace;">function(oldObject, key)</span> and the result used as the new value. In practice, the key argument is often ignored. For example,<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">"SUMMARY", function(o,k) { return 'name:' + o.name + ' date:' + o.date[0]; }</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
would mean to create a new SUMMARY property by concatenating two existing values.<br />
<br />
In many cases, you are still better off writing your own custom code. On further thought, I'm not sure how useful my module will be, since in JavaScript, it is just so <i>easy </i>to go<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">var newthing = {</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> newkey: oldObject.oldKey,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> ...</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">}</span><br />
<br />
Other times you can follow the conventions, or use the programmatic interface. However, if you want a quick way to "remodel" your domain object, this module might meet your needs. Let me know what you think. I think I'm going to try using this with ical-generator.<br />
<br />
<br />
<br />
<br />
<br />
<br />Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-77104174884060685012014-04-03T13:32:00.000-07:002014-04-03T13:34:46.666-07:00I'm a Web MagnateWell, one can dream. :-)<br />
<br />
I've spent the past few months developing a web site / app using a bunch of new (to me) stuff: <a href="http://nodejs.org/">node.js</a>, <a href="http://expressjs.com/">Express</a>, <a href="https://www.mongodb.org/">mongoDB</a>, along with HTML5, CSS3, and a little <a href="http://jquery.com/">JQuery</a> for a cool map using <a href="http://jvectormap.com/">jVectorMap</a>. All with JavaScript of course. Learned a lot along the way, and now feel somewhat competent with most of the technology. It's even hooked into Google Maps, Google Analytics, and <a href="http://www.addthis.com/">AddThis</a> for social networking.<br />
<br />
Check it out here at <a href="http://nextq.info/">nextq.info</a>. What is it? It is a search service to find upcoming <a href="http://en.wikipedia.org/wiki/Dog_agility">dog agility</a> trials. My main hobby.<br />
<br />
For hosting, I looked into a few cloud services such as Heroku and Nodejitsu, but decided to use <a href="https://modulus.io/">Modulus.io</a>. So far I've been <i>extremely </i>pleased. <br />
<ul>
<li>You get one month free, thereafter the "typical" $15 a month.</li>
<li>Their dashboard is cool-looking and, more importantly, fairly useful.</li>
<li>Creating and hooking up a small Mongo Database is simple, even a newbie like me can do it.</li>
<li>Uploading your code is simple, even a newbie like me can do it.</li>
<li>Linking their site to my domain name server was fairly straightforward, even a... (you get the idea).</li>
<li>On a crash it will send you an email (easy to configure) or text, then restarts.</li>
</ul>
<div>
There were a couple of minor gotchas. I forgot to route both nextq.info and www.nextq.info. It took me a little while to learn about the <span style="font-family: Courier New, Courier, monospace;">.modulusignore</span> file that tells the upload to ignore unnecessary or test files. (Though it is smart enough to ignore your <span style="font-family: Courier New, Courier, monospace;">node_modules</span> folder.)</div>
<div>
<br /></div>
<div>
So far so good. I especially enjoyed making up agility-centric custom 404 (and 400 and 500) pages. I'm finding that jVectorMap is a little flaky on some devices, so there is also a much simpler, somewhat "mobile friendly" search page. But the main parts seem to work fine and some of my agility friends are already using it.</div>
Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-80181835152593599422014-01-19T12:28:00.000-08:002014-01-19T12:28:13.938-08:00Yet Another Example Demo Node.js / Express ProjectI'm working on an app that finds upcoming events and locations, and wanted a quick link bring up the weather for that city. Which takes a little work. What I came up with make a very nice little node.js / express "example" or "demo" program.<br />
<br />
Thought I'd try it as a <a href="https://docs.google.com/presentation/d/1_3WsyM1wtgt6rTFCwVLnEyzzRfznkDwxLKHWE54cZK0/edit?usp=sharing">Google Docs presentation</a>.<br />
<br />
<a href="https://github.com/MorganConrad/node-express-wrd">Source code is on GitHub</a>. <br />
<br />
I'll likely add some more commentary here in the near future.Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-52140048722466224182013-12-19T15:17:00.000-08:002013-12-19T15:18:14.228-08:00From Node.js back to Java, Part 1: EventEmitter and CallbacksThere were a lot of features of Node.js and JavaScript that I liked. Why not bring them to Java? That's what I'm working on with <a href="https://github.com/MorganConrad/nava">Nava, "Bringing good ideas from node.js into Java"</a>. So far, I have implemented a fair amount. The code can be compiled against Java 6, Java 7, and Android 2.2, and presumably later versions of Android.<br />
<br />
The <b><a href="http://morganconrad.github.io/nava/javadocs/com/flyingspaniel/nava/emit/package-summary.html">emit package</a></b> is a pretty direct port of a <a href="http://nodejs.org/api/events.html#events_class_events_eventemitter">node.js EventEmitter</a> to Java, and can be used in a very similar manner as in node.js. The main difference is that the event handler cannot be a closure, (cause we don't <i>have </i>closures!) it must implement a simple interface, <a href="http://morganconrad.github.io/nava/javadocs/com/flyingspaniel/nava/emit/Emit.IListener.html">Emit.IListener</a>, with one method, <span style="font-family: Courier New, Courier, monospace;">handleEvent().</span> Producers of events would subclass or delegate to <a href="http://morganconrad.github.io/nava/javadocs/com/flyingspaniel/nava/emit/Emitter.html">Emitter</a>, much like the node class would extends or delegate to EventEmitter.<span id="goog_332084335"></span><span id="goog_332084336"></span><a href="http://www.blogger.com/"></a> This code essentially replaces Java's <a href="http://docs.oracle.com/javase/7/docs/api/javax/swing/event/EventListenerList.html">EventListenerList</a> and associated classes and interfaces. There are several major advantages:<br />
<br />
<ol>
<li><i>The code does not depend on Swing, so it could be used in Android.</i> </li>
<li>The "events" that get fired need not extend EventObject. They could be Strings or voids. However, in many cases, subclassing EventObject makes sense.</li>
<li>Generics are used to obviate the need for a lot of boilerplate, such as declaring a bunch of Listener subclasses and their respective delegation methods <span style="font-family: Courier New, Courier, monospace;">addFooListener()</span>, <span style="font-family: Courier New, Courier, monospace;">removeFooListener()</span>, etc...</li>
</ol>
<div>
There are also a few disadvantages, noted in the documentation. The main one is that everything is more loosely typed, as you'd expect as this is based upon JavaScript. Underneath, there are unchecked casts.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
The <b><a href="http://morganconrad.github.io/nava/javadocs/com/flyingspaniel/nava/callback/package-summary.html">callback </a></b>package is a rough port of JavaScript / node.js style callbacks. Your callback "closures" must implement the <a href="http://morganconrad.github.io/nava/javadocs/com/flyingspaniel/nava/callback/Callback.html">Callback </a>interface, or extend from <a href="http://morganconrad.github.io/nava/javadocs/com/flyingspaniel/nava/callback/AbstractCallback.html">AbstractCallback</a>. Callbacks can be "chained together", either manually using <span style="font-family: Courier New, Courier, monospace;"><a href="http://morganconrad.github.io/nava/javadocs/com/flyingspaniel/nava/callback/Callback.html#setNextCallback(com.flyingspaniel.nava.callback.Callback)">setNextCallback()</a></span>, or programatically with the utility method <a href="http://morganconrad.github.io/nava/javadocs/com/flyingspaniel/nava/callback/Callbacks.html#chainUp(com.flyingspaniel.nava.callback.Callback, com.flyingspaniel.nava.callback.Callback...)"><span style="font-family: Courier New, Courier, monospace;">Callbacks.chainUp()</span>.</a></div>
<div>
<br /></div>
<div>
You could then "submit" the callback directly by calling the first one, but that would happen synchronously, in-line, and, other than for initial testing, probably isn't worth it. The more "node-like" technique would be to create a <a href="http://morganconrad.github.io/nava/javadocs/com/flyingspaniel/nava/callback/CallbackExecutor.html">CallbackExecutor </a>(probably application-wide) and use <span style="font-family: Courier New, Courier, monospace;"><a href="http://morganconrad.github.io/nava/javadocs/com/flyingspaniel/nava/callback/CallbackExecutor.html#submitCallback(com.flyingspaniel.nava.callback.Callback, java.lang.Object, java.lang.Object...)">submitCallback()</a></span>, passing the first callback and it's data. Your thread will then continue, just like in node, with no need to await the completion (good) and no knowledge of the result (sometimes annoying).</div>
<div>
<br /></div>
<div>
For example, if you had a Reader that read a File into a String, and a Counter that counted words, the skeleton Java code would be:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> Reader reader = new Reader();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> Counter counter = new Counter();</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> Callbacks.chainUp(reader, counter );</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> CallbackExecutor cex = new CallbackExecutor(2,1);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> cex.submitCallback(reader, theFile);</span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> // your code then moves on (or just stops...)</span></div>
<div>
<br /></div>
<div>
As opposed to actual JavaScript/Node, which would look something like:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> fs.readFile(theFile, function(err, data) {</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> if (err) throw err;</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> // normally the counting code would be in-line here</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> // but for modularity similar to the Java</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> countWords(data);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> });</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> // your code then moves on (or just stops...)</span></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
There's more, but that's a start for today. Check out the <a href="https://github.com/MorganConrad/nava/blob/master/README.md">readme </a>file, <a href="http://morganconrad.github.io/nava/javadocs/">javadocs</a>, and <a href="https://github.com/MorganConrad/nava/tree/master/test/com/flyingspaniel/nava">JUnit tests</a> for more information.</div>
<div>
<br /></div>
Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com1tag:blogger.com,1999:blog-6276165447167620036.post-8133516762346479312013-12-05T10:18:00.000-08:002013-12-05T10:18:10.904-08:00Javadocs on GitHub for DummiesGitHub has a feature to allow you to <a href="http://pages.github.com/">upload supporting documents</a>, such as Javadocs, into a gh-pages branch, for easy access by people browsing your project. The problem is that it's a huge PITA to configure, especially if you are new to git. Here's how I did it. This is for a Windows machine and a typical java project. YMMV.<br />
<br />
For example, if you are working on ProjectFoo in GitHub, somewhere there will be a ProjectFoo folder on your local disk that contains a README.me, .gitignore, and folders such as src and test. I will assume that this has been done, plus your project, in this state, already exists on GitHub as <span style="font-family: Courier New, Courier, monospace;">github.com/UserName/ProjectFoo.git</span>.<br />
<br />
You want your javadocs to go into a <i>completely separate folder from your main project</i>. Otherwise you are <a href="http://stackoverflow.com/questions/11938270/how-i-can-move-javadoc-from-master-branch-to-gh-pages-branch-on-github-using-egi">constantly switching back and forth</a>. I have organized my projects as follows, splitting my "master" GIT_FOLDER into master and gh-pages subfolders:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"> GIT_FOLDER (all git projects go here)</span><br />
<span style="font-family: Courier New, Courier, monospace;"> master</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ProjectFoo</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ProjectSomeOther</span><br />
<span style="font-family: Courier New, Courier, monospace;"> gh-pages</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ProjectFoo</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ProjectSomeOther</span><br />
<br />
You don't have to follow this arrangement. But the important thing is to get the javadocs separated from the main project so the branches don't keep stepping on each other.<br />
<br />
<h3>
Initial Setup</h3>
<span style="font-family: Courier New, Courier, monospace;">cd to GIT_FOLDER/gh-pages</span> (or, if you have a different setup for the gh-pages branch, to the corresponding folder) and open a Command Window.<br />
<br />
Checkout your main branch there<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">>git clone https://github.com/UserName/ProjectFoo.git</span><br />
<br />
This will create a folder GIT_FOLDER/gh-pages/ProjectFoo. CD there and check your branch.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">>cd ProjectFoo. </span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;">>git branch</span><br />
<span style="font-family: Courier New, Courier, monospace;">* master</span><br />
<br />
Do not create a gh-pages branch yet! Instead, checkout a new <i>orphan </i>branch named gh-pages<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">>git checkout --orphan gh-pages</span><br />
<span style="font-family: Courier New, Courier, monospace;">Switched to a new branch 'gh-pages'</span><br />
<br />
In my experience, git still says you are on master if you go git branch, but ignore that for now, it will eventually figure things out.<br />
<br />
A directory command (<span style="font-family: Courier New, Courier, monospace;">dir /A /B</span>) should show your Java code, with src and possibly test folders, something like:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.git</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.gitignore</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">LICENSE</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">README.md</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">src</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">test</span><br />
<br />
All you want here are the javadocs.<br />
<ol>
<li>Delete the LICENSE file, and the src and test folders.</li>
<li>Create (or copy) the javadocs to a folder named javadocs.</li>
<li>Double check that .gitignore isn't doing anything too goofy</li>
</ol>
<div>
Your folder should now contain:</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.git</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">.gitignore</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">javadocs</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">README.md</span></div>
<div>
<br /></div>
<div>
Check your <span style="font-family: Courier New, Courier, monospace;">git status</span>. It should now have the correct branch. Your list of deleted files will vary.</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">>git status</span></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># <i>On branch gh-pages</i></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># Changes not staged for commit:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># (use "git add/rm <file>..." to update what will be committed)</file></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># (use "git checkout -- <file>..." to discard changes in working directory)</file></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">#</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># deleted: LICENSE</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># deleted: src/com/company/package/SomeFile1.java</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># deleted: src/com/company/package/SomeFile2.java</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># deleted: src/com/company/package/package-info.java</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># deleted: test/com/company/package/SomeFile1Test.java</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">#</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># Untracked files:</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># (use "git add <file>..." to include in what will be committed)</file></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">#</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># javadocs/</span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
Add javadocs to the repository. You may see warnings about line endings.</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">>git add javadocs</span></div>
<div>
<br /></div>
<div>
Commit them. You may see warnings about line endings.</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">>git commit -m "1st checkin of javadocs"</span></div>
<div>
<br /></div>
<div>
<div>
Now, commit everything else to delete the src and test folders</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">>git commit -a -m "deleted src from gh-pages"</span></div>
</div>
<div>
<br />
As a final check, git status should now be clean and git branch shows the new branch:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">>git status</span><br />
<span style="font-family: Courier New, Courier, monospace;"># On branch gh-pages</span><br />
<span style="font-family: Courier New, Courier, monospace;">nothing to commit, working directory clean</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;">>git branch</span><br />
<span style="font-family: Courier New, Courier, monospace;">* gh-pages</span><br />
<span style="font-family: Courier New, Courier, monospace;"> master</span><br />
<br /></div>
<div>
Finally, push to GitHub. </div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">>git push origin gh-pages</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Username for 'https://github.com': UserName</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Password for 'https://</span><span style="font-family: 'Courier New', Courier, monospace;">UserName</span><span style="font-family: Courier New, Courier, monospace;">@github.com':</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Counting objects: ...</span><br />
<span style="font-family: 'Courier New', Courier, monospace;">... reused 0 (delta 0)</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">To https://github.com/UserName/project.git</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> * [new branch] gh-pages -> gh-pages</span></div>
</div>
<div>
<br /></div>
<div>
<div>
Now, you can login to GitHub with your browser, and you should see the new branch. Assuming you want a prominent link to these javadocs, go to the main branch and add this line somewhere in the README file:</div>
</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">[JavaDocs are here](http://UserName.github.io/ProjectFoo/javadocs/)</span></div>
<div>
<br /></div>
<div>
<span style="font-family: inherit;">For an example, see <a href="https://github.com/MorganConrad/nava">my Nava project</a>.</span><br />
<h3>
<span style="font-family: inherit;"><b>Later on, how to update the JavaDocs:</b></span></h3>
<br />
<ol>
<li>Generate (or copy) them into that javadocs folder</li>
<li>Open a command window there</li>
<li><span style="font-family: Courier New, Courier, monospace;">git status</span> to see whats going on. <i>You should be on the gh-pages branch</i>!</li>
<li>You will probably need to add some new docs: <span style="font-family: Courier New, Courier, monospace;">git add .</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">git commit -m "some comment"</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">git status</span> (if you are paranoid)</li>
<li><span style="font-family: Courier New, Courier, monospace;">git push origin gh-pages</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">git status</span> Everything should be clean.</li>
</ol>
<div>
<br /></div>
</div>
Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-86345704897216482222013-11-20T10:33:00.000-08:002013-11-20T10:33:03.484-08:00Leaping into node.js and JavaScript, the ConculsionIn three weeks I've gone from <a href="http://flyingspaniel.blogspot.com/2013/10/leaping-into-nodejs-and-javascript-part1.html">node.js newbie</a> to <a href="http://flyingspaniel.blogspot.com/2013/11/leaping-into-nodejs-and-javascript-part.html">beginner </a> to having a <a href="https://npmjs.org/package/fcs">module on npm</a>. Despite the off-beat field of endeavor (Flow Cytometry), it has dozens of downloads. I think some are robots harvesting the net.<div>
<br /></div>
<div>
<b>Things I liked:</b><div>
<ul>
<li><a href="http://www.jetbrains.com/webstorm/">WebStorm </a>made working with GitHub simple and painless. No more series of three commands to add files, commit locally, and push to the remote site.</li>
<li>Integration with <a href="https://travis-ci.org/">Travis-ci</a> was <a href="http://about.travis-ci.org/docs/user/getting-started/">nearly painless</a>. The main gotcha was in pasting the cute little "the build is passing" icon back into the GitHub readme. <i>Capitalization matters</i>. If your repository is CamelCase, the links to travis-ci must be CamelCase.</li>
<li>The package.json file and conventions. Simpler, and much less verbose, than your typical web.xml descriptor. Use <span style="font-family: Courier New, Courier, monospace;"><a href="https://npmjs.org/doc/init.html">npm init</a></span> to create a good template, and then it's simple to edit by hand. I found the <a href="https://npmjs.org/doc/json.html">online docs</a> fairly inscrutable. Better to look at various package.jsons from other npm projects, then reread the man pages with the benefit of examples.</li>
<li>Publishing to npm was also painless. My preference is to use a text editor, not npm commands, to edit the package.json file. Once you have logged in for the first time to npm, just go <span style="font-family: Courier New, Courier, monospace;"><a href="https://npmjs.org/doc/publish.html">npm publish</a> </span></li>
<li><span style="font-family: inherit;">Using simple JavaScript</span><span style="font-family: inherit;"> "objects" { } to hold incoming options, and to return multiple values from a method.</span></li>
<li><span style="font-family: inherit;">JavaScript "short-circuit or" syntax to test for nulls and use defaults. <i>e.g.</i> </span><span style="font-family: Courier New, Courier, monospace;">encoding = options.encoding || 'utf8';</span></li>
<li>JavaScript's first class functions, combined with <a href="http://nodejs.org/api/buffer.html">node.js Buffers</a>, were superb in reading binary data from a file, handling various data sizes (16, 32, 64 bit) and <a href="http://en.wikipedia.org/wiki/Endianness">endianness</a>.</li>
<li>JavaScript's optional function arguments, <i>most </i>of the time.</li>
</ul>
<div>
<b>Things that worked "o.k.":</b></div>
</div>
<div>
<ul>
<li>All the node.js callbacks. It is a huge mental shift. A few times I got stuck with just too many callbacks to figure things out. Still don't know how to <a href="http://stackoverflow.com/questions/19918326/how-do-you-read-a-file-or-stream-synchronously-in-node-js#comment29640751_19918326">read a Stream synchronously</a>, since the basic <span style="font-family: Courier New, Courier, monospace;">rs.read()</span> method doesn't work. Rethinking or refactoring so that the callbacks were distributed across more methods helped. In other words, instead of one function with three callbacks, consider refactoring into three functions, each with one callback.</li>
<li>Closures. O.K., yes, they are nice. For node.js, required. But writing line after line ending with <span style="font-family: Courier New, Courier, monospace;">function() {</span> and then adding all the <span style="font-family: Courier New, Courier, monospace;">});</span> at the end, and getting it <i>right</i>, gets old. And verbose. And these closures in general have no names, so they lose out as self-documenting.</li>
<li>Because it seemed cool and trendy, I used <a href="http://visionmedia.github.io/mocha/">mocha </a>for unit tests, instead of the more "JUnit-like" <a href="https://github.com/caolan/nodeunit">nodeunit</a>. But I couldn't run it well from WebStorm 7.0. Lo and behold, one of the features of WebStorm 7.0.2 is improved support for Mocha! Problem largely solved, I can run my unit tests from the IDE. Code coverage is still an issue.</li>
<li>Not sure I'm a fan of Mocha's "literate" style. In <a href="http://junit.org/">JUnit</a>, I'd name my test method something like <span style="font-family: Courier New, Courier, monospace;">testReadFCSFile()</span>. In Mocha, you use <span style="font-family: Courier New, Courier, monospace;">describe </span>to wrap the anonymous function, e.g. <span style="font-family: Courier New, Courier, monospace;">describe('Read an FCS File', function() ...) </span> JUnit is simpler and more concise. OTOH, Mocha definitely encourages you to think about what the results should be.</li>
</ul>
<div>
<b>Things that didn't work or were annoying:</b></div>
</div>
</div>
<div>
<ul>
<li>"<span style="font-family: Courier New, Courier, monospace;">this</span>" changes in a closure. So you have to go <span style="font-family: Courier New, Courier, monospace;"> self = this</span>, and <i>remember </i>to use <span style="font-family: Courier New, Courier, monospace;">self</span>!</li>
<li>JavaScript thinking that it "O-O", but the terminology is pretty fuzzy and loose. For example, those handy { } thingies that just hold a few snippets of data - what <i>do </i>you call them? "Object" is just plain <i>wrong</i>, since they hold no real <i>behavior</i>, and "hash", (or "Map" or "Dictionary" or even "Key/Value pairs") which I greatly prefer, doesn't seem all that standardized. Let's push for "hash" or "map"!</li>
<li>The documentation for setting up your test scripts is all Unix based, not Windows. For example, their script suggested for mocha testing is <span style="font-family: Courier New, Courier, monospace;">"test": "./node_modules/.bin/mocha"</span> which doesn't work at all under Windows. I played around for an hour and found a workaround, "<span style="font-family: Courier New, Courier, monospace;">node node_modules/mocha/bin/mocha</span>". Turns out that there is a super-simple way, <span style="font-family: Courier New, Courier, monospace;">"test":"mocha". </span>But I had to find this out by <a href="http://stackoverflow.com/questions/20013989/configure-node-npm-package-json-so-that-npm-test-works-on-both-unix-and-window/20014206#20014206">asking on StackOverflow</a>.</li>
<li>I still haven't figured out how to get test coverage for mocha. Looks like I might need to install Karma, then Istanbul, then whatever... More to come. I'm concerned that just as Windows has <a href="http://en.wikipedia.org/wiki/DLL_Hell">DLL Hell</a> and Java has <a href="http://en.wikipedia.org/wiki/JAR_hell#JAR_hell">JAR hell</a>, node may turn into NPM hell.</li>
<li>The documentation is still, er, "young". Maybe as you expect, node.js is only in version 0.10. A lot of times you have to make educated guesses, or look via the debugger, as to what events a Stream will emit. Looking at the node source is helpful, but it is daunting for a JavaScript beginner and there are distressingly few comments in there. Many are of the "here be Dragons" type around tricky code, not "here be the parameters". Java and Javadocs are far superior here.</li>
<li>At least for methods, I really miss Java's "verbose" style, where you know the types of the arguments and the return value. And the JavaDocs. Too often in JavaScript / node you are guessing, or, once you are experienced, using your instincts, as to whether an argument is a String, a hash, or an object. And often the answer is "all of the above".</li>
<li>A common Java practice is to have an "all-powerful" method (or constructor) with all possible parameters, and "convenience" methods that provide some default parameters. Currying. This proves to be very awkward in JavaScript, with ugly code like </li>
</ul>
<div>
<span style="font-family: Courier New, Courier, monospace;"> [].unshift.call(arguments, moreArgs);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> allPowerful.apply(this, arguments);</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<br /></div>
</div>
Morgan Conradhttp://www.blogger.com/profile/15963409751532704983noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-64573526431136743632013-11-15T09:08:00.000-08:002013-11-20T10:39:50.784-08:00Leaping into node.js and JavaScript, part deuxWell, I'm making a lot of progress since my <a href="http://flyingspaniel.blogspot.com/2013/10/leaping-into-nodejs-and-javascript-part1.html">post of two weeks ago</a>. Might be starting to get the hang of this JavaScript and node.js stuff. First, a few followups from last post.<br />
<br />
<br />
<ol>
<li>I like <a href="http://www.jetbrains.com/webstorm/">WebStorm</a>. Well worth the small fee for a personal license. Even just the way it <a href="http://flyingspaniel.blogspot.com/2013/10/i-dont-get-git-and-github.html">makes Git and GitHub painless</a> is almost worth the price. And for a relative newbie blundering along it's great. Especially since a lot of the node.js documentation is a bit sketchy. Setting breakpoints and looking at fields is wonderful. My only complaint is that sometimes it gets very very slow. I'm using version 7.0. There's a 7.0.1 upgrade available, not sure what it fixes.</li>
<li>The SAMs book <i><a href="http://www.amazon.com/Sams-Teach-Yourself-Node-js-Hours/dp/0672335956">Teach Yourself node.js in 24 Hours</a> </i>has been pretty useful - much better than many of the "in 24 Hours" books.</li>
<li><a href="http://www.amazon.com/Professional-Node-js-Building-Javascript-Scalable/dp/1118185463" style="font-style: italic;">Professional Node.js: Building JavaScript Based Scalable Software</a> is o.k., but a bit disappointing compared to most of the other <a href="http://www.wrox.com/WileyCDA/">WROX </a>books I have read.</li>
</ol>
<div>
A few more resources I have found that seem useful:</div>
<div>
<ol>
<li><a href="http://shop.oreilly.com/product/9780596805531.do">JavaScript the Definitive Guide</a> is essential. Get it.</li>
<li>Manuel Kiessling is developing a <i><a href="https://leanpub.com/nodecraftsman">Node Craftsman</a> </i>followup book. I't still pretty early in development but might be useful. </li>
<li>A free download of <i><a href="http://it-ebooks.info/book/274/">JavaScript the Good Parts</a></i>. I'm temporarily ignoring some of his advice, but it's still a good reference.</li>
<li>Of course, <a href="http://stackoverflow.com/">Stackoverflow</a>, within limits. Many of the responses are very client-HTML-ish (not node.js serverish) and not all are good. But you can dig to find good information.</li>
</ol>
<div>
<b>Javascript</b></div>
<div>
<br /></div>
<div>
I'm coming from 20+ years of Java experience, and my JavaScript shows it. I put in semicolons. Looping over arrays I use a for-next with indices, not <span style="font-family: Courier New, Courier, monospace;">each()</span>. I'd like to use <span style="font-family: Courier New, Courier, monospace;">for (var x in theArray)</span>more, except that stupidly returns all the <i>indices </i>in the array, not all the <i>values</i>. Sorry, <i>makes no sense for an array</i>, but since <a href="http://javascript.about.com/od/problemsolving/a/notarrays.htm">JavaScript arrays aren't real</a> I understand whats going on. But still been burned there several times. And I still often mistype my loops as <span style="font-family: Courier New, Courier, monospace;">for (<b>int</b> i=0; i<....)</span>. Where the "int" should be "var". Of course, when I go back to programming in Java I'll surely make the opposite mistake. :-)</div>
<div>
<br /></div>
<div>
Creating a JavaScript <b>class </b>is fraught with way too much danger. There are too many ways to do it, all the examples are different, and you can run into religious wars. Frankly, I think a lot of the people writing have no clue about OOP classes. I ended up using <a href="http://www.crockford.com/javascript/inheritance.html">"classical" style</a>, partly cause it worked well with node's <a href="http://en.wikipedia.org/wiki/CommonJS">CommonJS </a>module structure to simplify the namespace and export issues, and mainly cause it felt most natural to me. With more experience this may change. Classical style uses the .prototype field a lot. It is well described in <i><a href="http://shop.oreilly.com/product/9780596805531.do">JavaScript the Definitive Guide, 6th ed</a></i>. in section 9.3, "Java-Style Classes in JavaScript". Example classical style code below:<br />
<br />
<script src="https://gist.github.com/MorganConrad/7487819.js"></script>
I'll talk more about working with node.js in future posts...</div>
</div>
Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-44981107662142863792013-10-29T10:56:00.001-07:002013-10-29T13:00:48.731-07:00Leaping into node.js and JavaScript, part1A close friend with tons of tech experience had been urging me to learn JavaScript. He uses it all the time and loves it. Last week the local library had a Sams book, <i><a href="http://www.amazon.com/Sams-Teach-Yourself-Node-js-Hours/dp/0672335956">Teach Yourself node.js in 24 Hours</a></i>. So I checked it out. How hard could it be? :-) I have over twenty years experience in Java, and more before that in C, C++, Pascal etc. All your classic <a href="http://en.wikipedia.org/wiki/Strong_and_weak_typing">strongly typed</a>, O-O / procedural languages. But I have zero experience in node.js, or anything.js, and JavaScript. Let the adventure begin.<br />
<br />
I always had the bias that JavaScript was a "toy language" just for browsers. At least the second half is wrong. The node.js guys took <a href="https://code.google.com/p/v8/">Google's Chrome JavaScript runtime</a> engine out of the browser, so you can run JavaScript, for example, from the command line. Just like a JRE lets you run Java. Cool. There is an npm installer to search for and download additional libraries. It's very quick to get a toy "Hello World" HTTP <a href="https://github.com/shapeshed/nodejsbook.io.examples/blob/master/hour01/example01/server.js">server up and running</a>. So far, so good.<br />
<br />
A couple of problems.<br />
<br />
Many of the examples are of the silly "Hello World" variety. Data is collected all well and good, and then they use console.log(theData) to output to the console. Now, very few real-world apps are going to do that. More importantly, doing it this way glosses over some issues with all the callbacks that are the bane, or blessing, of node.js. (For a good time, do a Google search on "node.js cancer") Depends on your brain. In a real application, you want to pass this data on to something else, such as an HTTP reply, an HTTP/XML/JSON parser, etc... And this gets tricky. Simply logging to the console obfuscates some of the key themes of node.js control flow.<br />
<br />
Many generic intro Javascript blogs or books have crappy code too. All the code is glommed into a single huge file, variables are named "$", and prints to the console happen. <b>This is not proper software engineering</b>! Javascript / node.js may be the hot new nailgun that lets you build sites quickly and easily, but you still need to put studs in the wall and use electrical boxes.<br />
<br />
I found the following site pretty useful: <a href="http://javascriptissexy.com/how-to-learn-javascript-properly/">How to Learn Javascript Properly</a>, along with <a href="http://javascriptissexy.com/learn-node-js-completely-and-with-confidence/">Learn Node.js Completely and with Confidence</a>. He's opinionated but I agree with a lot of them.<br />
<br />
He suggests that you get "the absolute best editor (IDE) for programming JavaScript", <a href="http://www.jetbrains.com/webstorm/">JetBrains' WebStorm</a>. I agree completely. It's free for 30 days, thereafter very affordable. Don't know about you, but my time and sanity is worth way over $49. And for a "traditional" Java programmer moving over it's great to be able to have some syntax help and a nice graphical debugger. JavaScript is one of those loosely typed everything is an Object languages. So many times opening up a variable to see "what the heck type is this?" and "what fields can I access?" is simply the best way to blunder along.<br />
<br />
I had separately discovered the <a href="https://leanpub.com/">Leanpub </a>online ebook, <i><a href="http://www.nodebeginner.org/">The Node Beginner Book</a></i> by Manuel Kiessling, which you can buy along with <a href="https://leanpub.com/hands-on-nodejs">Hands-on Node.js</a> in a <a href="https://leanpub.com/b/node">$9.99 bundle</a>. Do so. So far I found both useful. They actually talk a bit about how to structure your source code for a realistic application. Not dumping everything into one huge file with $s as variable names!<br />
<br />
Anyway, since he also recommends these books, another tip of the hat to him. Since I like what's he's saying so far, I took his advice and ordered a more advanced book, <i><a href="http://www.amazon.com/Professional-Node-js-Building-Javascript-Scalable/dp/1118185463">Professional Node.js: Building JavaScript Based Scalable Software</a>. </i>I'll let you know what I think once it arrives.<br />
<br />
Finally, stealing a phrase from <a href="http://mindviewinc.com/Index.php">Bruce Eckel</a>, a word about "Thinking in node.js". Code flow does not flow in the normal sense. There are callbacks. You don't have an option. <i>If you can't buy into or grok the callbacks, don't use node.js</i>. It's gonna take me a while... For example, here was some of my first, ignoramus code. (It's using a <a href="https://github.com/mikeal/request">popular request library</a> to do basic http requests)<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">var googleBody = 'not there';</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Request.get('http://www.google.com', function (error, response, body) {</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> googleBody = body;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">});</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">doSomethingWith(googleBody); // like parsing the HTML...</span><br />
<br />
This doesn't work! You have to remember that the callback executes at some indefinite time in the future, so the <span style="font-family: Courier New, Courier, monospace;">googleBody </span>passed to <span style="font-family: Courier New, Courier, monospace;">doSomethingWith </span>is very likely to be 'not there'. I really should have known this, and, when I realized it a couple of hours later after some struggles, it was a real Homer Simpson "Doh" moment. But I'm learning. More to come.Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-68977921324379272512013-10-08T13:49:00.001-07:002013-10-29T09:59:36.397-07:00I don't get git and githubThe whole add / commit / push throws me. When I first described the system to my wife, a former techwriter for computer networking companies, she said "geez, that sounds strange and clunky, is that something from UNIX?". <a href="http://en.wikipedia.org/wiki/Git_(software)">Well, it is</a>. But I think I can adjust...<br />
<br />
Anyway, what has me really stymied is how to publish javadocs to <a href="http://en.wikipedia.org/wiki/GitHub">GitHub</a> in a sane matter. GitHub allows you can create a branch, hardcoded to "gh-pages", to hold docs. But the problem is how to move up to date documents (generated manually, via ant, etc.) from some local folder into the proper branch of the GitHub repository.<br />
<br />
I'm not the only one. <a href="http://stackoverflow.com/questions/11938270/how-i-can-move-javadoc-from-master-branch-to-gh-pages-branch-on-github-using-egi">Here is a 15 step process from a question on StackOverflow</a>. Wow!<br />
<br />
Here's a very similar StackOverflow question <a href="http://stackoverflow.com/questions/8145927/managing-code-documentation-in-the-same-git-directory-structure">with a complex answer</a>. May be the best. Involves a symbolic link between the second branch and the internal folder which is something I was considering. Note that the question was raised two years ago and the answer hasn't been "confirmed". And both questioner and answerer have a ton of experience (StackOverflow points). There aren't any "oh yeah baby, that's right" comments. So it's not like a bunch of smart people agreed that this was a wonderful solution.<br />
<br />
Here is <a href="http://dephraser.net/blog/2012/04/16/ant-javadox-and-github/">yet another solution</a>. Ant is nice but not required. However, it does involve checking out one branch (the javadoc .html files) over the code branch, wiping out the .java code files (even though one might call this "reverting", they are gone, right?) which seems, well, bizarre. Frankly, the whole branching / wiping out thing strikes me as bizarre And the generated javadocs then get removed when you return to the main branch and checkout the code. So you can't look at them anymore.<br />
<br />
I think the seconds answer, with two folders, one per branch, with a link is best.<br />
<br />
Question in general for git experts: do you usually keep branches in separate folders / separate clones? Or do you throw them all together into one directory and trust git in overwriting all of your previous work? Maybe I'm just too old fashioned and paranoid? :-)<br />
<br />
<br />
<br />Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-52240327432248222622012-09-06T14:49:00.002-07:002016-11-10T09:12:16.604-08:00ExecutorService, Part 2, the "Unhappy Path"In a <a href="http://flyingspaniel.blogspot.com/2012/06/how-to-parallelize-your-algorithm-with.html">previous post</a>, I explained how to parallelize your algorithms using an <a href="http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html">ExecutorService</a>. Here we consider the unhappy path when your algorithms throw Exceptions, or you wish to stop their execution. This requires adding a few lines of code (a try - catch block) to the <span style="font-family: "courier new" , "courier" , monospace;">calculateInParallel( )</span> method.<br />
<br />
<br />
<pre class="language-java"><code>
try {
List<Future<CallableCalculation>> results = myService.invokeAll(tasks);
for (Future<CallableCalculation> future : results) {
CallableCalculation cc = future.get();
resultMap.put(cc.year, cc.getPercentileResults());
}
}
catch (ExecutionException ee) {
// what to do?
}
catch (InterruptedException ie) {
// what to do???
}
</code></pre>
<span style="font-family: "georgia" , "times new roman" , serif;">But "what to do"? You probably want to shut down the </span><span style="font-family: "courier new" , "courier" , monospace;">ExecutorService </span><span style="font-family: "georgia" , "times new roman" , serif;">and return some status for failure or interruption. To handle this, plus some of the other "utility" methods like calculating a "good" number of threads to use, I wrote a class </span><span style="font-family: "courier new" , "courier" , monospace;">CallablesCaller</span><span style="font-family: "georgia" , "times new roman" , serif;">, which handles this common utility work, (in case of failure it returns </span><span style="font-family: "courier new" , "courier" , monospace;">null </span><span style="font-family: "georgia" , "times new roman" , serif;">for the List of results, and also has a </span><span style="font-family: "courier new" , "courier" , monospace;">wasInterrupted() </span><span style="font-family: "georgia" , "times new roman" , serif;">method), while preserving the key idea of submitting a list of </span><span style="font-family: "courier new" , "courier" , monospace;">Callables </span><span style="font-family: "georgia" , "times new roman" , serif;">to an </span><span style="font-family: "courier new" , "courier" , monospace;">ExecutorService</span><span style="font-family: "georgia" , "times new roman" , serif;">. The source code (two files, you also need </span><span style="font-family: "courier new" , "courier" , monospace;">ConcurrentUtil</span><span style="font-family: "georgia" , "times new roman" , serif;">) is <a href="https://www.assembla.com/code/hastur/subversion/nodes/trunk/Hastur_A22/src/com/flyingspaniel/concurrent">available here</a>, and the <a href="https://www.assembla.com/code/hastur/subversion/nodes/trunk/Hastur_JUnit/src/com/flyingspaniel/concurrent/CallablesCallerTest.java">unit test is here</a>.</span>
<br />
<span style="font-family: "georgia" , "times new roman" , serif;"><br /></span>
<span style="font-family: "georgia" , "times new roman" , serif;">Enjoy, and I hope you find this useful in your efforts. Note that </span><span style="font-family: "courier new" , "courier" , monospace;">CallablesCaller </span><span style="font-family: "georgia" , "times new roman" , serif;">does <i><b>not </b></i>do anything fancy to shutdown immediately if any of the </span><span style="font-family: "courier new" , "courier" , monospace;">Callables </span><span style="font-family: "georgia" , "times new roman" , serif;">fail. This is because it uses the simple, basic </span><span style="font-family: "courier new" , "courier" , monospace;">invokeAll()</span><span style="font-family: "georgia" , "times new roman" , serif;"> method. If you wanted to stop ASAP on a failure, you'd want to add on an <a href="http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorCompletionService.html">ExecutorCompletionService</a> so that you could poll the results as they arrive and handle failures immediately. I didn't add this complication because the algorithms I am using are simple statistics that hardly ever fail, and, even if one does fail, the other results are still useful.</span>Morgan Conradhttp://www.blogger.com/profile/15963409751532704983noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-86448020400561592952012-07-02T16:52:00.003-07:002013-11-20T10:40:22.925-08:00Java Event Handling, Revisited Quite a while ago I blogged about replacement code for Java's javax.swing.event.<a href="http://docs.oracle.com/javase/7/docs/api/javax/swing/event/EventListenerList.html">EventListenerList</a>, which never appealed to me. (Blog posts are <a href="http://flyingspaniel.blogspot.com/2009/12/java-event-handling-part-3-double.html">Here</a>, <a href="http://flyingspaniel.blogspot.com/2009/12/swing-event-handling-part-2.html">here</a>, and <a href="http://flyingspaniel.blogspot.com/2009/12/swing-event-handling-possible.html">here</a>) The EventListenerList code, is, well, <i>icky</i>. You jump by 2 along an array of mixed Classes and Listeners, and non-reassuring comments in the source code like "it provides ... <span style="background-color: white;">a <i>degree </i>of MT (multi-threaded) safety</span><span style="background-color: white;"> (<i>when used correctly</i>)." It was written before the newfangled <span style="font-family: 'Courier New', Courier, monospace;">java.util.concurrent</span> code, such as the <a href="http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CopyOnWriteArraySet.html">CopyOnWrite </a>collections, which were written by experts and are <i>perfect </i>for this task. </span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"> Also, to use EventListenerList, you are required to define a little Listener class for each type of event to be fired, which is repetitive boilerplate work. Now that Java has Generics this is less necessary. EventListenerList will only handle events that implement the marker interface EventListener. Which isn't horrible, but, if you want a more general purpose class to "fire" something else, say a String, to listeners, it won't do the job. Finally, </span>
EventListenerList <span style="background-color: white;">depends on Swing. If your project targets Android, you don't have Swing. So, if you like (or are just plain used to) the concept of a decoupled event listener, but are looking for something a bit easier to use and with broader application, this code might be for you.</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"> I have slightly updated and reorganized the code, which is available on <a href="http://www.assembla.com/">Assembla</a>. The <a href="http://www.assembla.com/code/hastur/subversion/nodes/trunk/Hastur_A22/src/com/flyingspaniel/event">source code is here</a>, and the <a href="http://www.assembla.com/code/hastur/subversion/nodes/trunk/Hastur_JUnit/src/com/flyingspaniel/event">JUnit tests here</a>.</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"> <span style="font-family: 'Courier New', Courier, monospace;"><b><a href="http://www.assembla.com/code/hastur/subversion/nodes/trunk/Hastur_A22/src/com/flyingspaniel/event/Events.java">Events.java</a></b></span> defines three inner interfaces. In the future, I may add some utility methods here to manipulate events.</span><br />
<br />
<ol>
<li><span style="background-color: white;"><span style="font-family: 'Courier New', Courier, monospace;"><b>Events.Listener<e></e></b></span> defines a listener, generically, based upon the type of event it listens for. No need to create an actually boilerplate class for each different listener.</span></li>
<li><span style="background-color: white;"><span style="font-family: 'Courier New', Courier, monospace;"><b>Events.Broadcaster<e></e></b></span> is a convenience interface for any class that fires a <i>single type</i> of event. You don't need to define and implement it (Java never defined such as interface) but it could be useful.</span></li>
<li><span style="background-color: white;"><span style="font-family: 'Courier New', Courier, monospace;"><b>Events.Broadcasters</b></span> is a convenience interface for any class that fires <i>multiple types</i> of events.</span></li>
</ol>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"><b><a href="http://www.assembla.com/code/hastur/subversion/nodes/trunk/Hastur_A22/src/com/flyingspaniel/event/Broadcaster.java">Broadcaster</a> </b></span>is an implementation of Events.Broadcaster to fire a single type of event. In the common case where there are 0 or 1 listeners, it just uses a direct reference, aListener. When there are multiple listeners, it uses a CopyOnWriteArraySet. Note that this avoids the common practice of adding the same listener twice, which, in my experience, is almost always a bug.</div>
<div>
<br /></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"><b><a href="http://www.assembla.com/code/hastur/subversion/nodes/trunk/Hastur_A22/src/com/flyingspaniel/event/Broadcasters.java">Broadcasters</a> </b></span>is an implementation of Events.Broadcasters. It can fire multiple types of event, using a HashMap to redirect them by the specific class of the event. Note that it uses == logic, <i>not </i><span style="font-family: 'Courier New', Courier, monospace;">instanceof</span>. If you want to subclass an event (in my experience, extremely rare - look at the built in Java events) you'll have to do something clever.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
One drawback of this concept is that, due to erasure, a class cannot listen to two types of events. That is, you cannot go</div>
<div>
<br /></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">public class FooBarListener implements Events.Listener<foo>, Events.Listener<bar> </bar></foo></span></div>
<div>
<br /></div>
<div>
One option is to implement Double Dispatch, as illustrated in <a href="http://www.assembla.com/code/hastur/subversion/nodes/trunk/Hastur_JUnit/src/com/flyingspaniel/event/DoubleDispatchUnitTest.java">DoubleDispatchUnitTest</a>. I find double-dispatch confusing, so the simpler alternative is to define your separate listeners in inner classes, e.g.</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">public class FooBarListener {</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> private Events.Listener<foo> fooListener = new Events.Listener<foo>() {</foo></foo></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> public void handleEvent(Foo fooEvent) {</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> // do stuff here</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> }</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> };</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> private Events.Listener<bar> barListener = new Events.Listener<bar>() {</bar></bar></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> public void handleEvent(Bar barEvent) {</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> // do stuff here</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> }</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"> };</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">}</span></div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"><br /></span>Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-29579922922391210662012-06-29T13:54:00.000-07:002013-10-29T13:03:37.267-07:00From java.awt.print.Pageable to PDF, RevisitedIn an <a href="http://flyingspaniel.blogspot.com/2012/04/from-javaawtprintpageable-to-pdf.html">earlier post</a>, I presented code to "print" to a PDF file from a <a href="http://docs.oracle.com/javase/6/docs/api/java/awt/print/Pageable.html">Pageable</a>. Since then I have added the ability to concatenate <i>multiple </i>Pageables into a single PDF file, with the option for bookmarks. The previous code was modified and some things renamed.<br />
<br />
A small interface, <span style="font-family: 'Courier New', Courier, monospace;">PDFBookmarker, </span>was added.<br />
The previous class, <span style="font-family: 'Courier New', Courier, monospace;">PDFPrinter</span>, was renamed <span style="font-family: 'Courier New', Courier, monospace;">PDFStream </span>(mainly to keep it different). The new class takes an <a href="http://docs.oracle.com/javase/6/docs/api/java/io/OutputStream.html"><span style="font-family: 'Courier New', Courier, monospace;">OutputStream </span></a>in the constructor, since it may be shared across multiple prints.<br />
The key method, which was called <span style="font-family: 'Courier New', Courier, monospace;">printToPdf()</span>, is renamed to the clearer <span style="font-family: 'Courier New', Courier, monospace;">appendToPDF()</span>.<br />
A few new methods, such as <span style="font-family: 'Courier New', Courier, monospace;">newPage()</span> and <span style="font-family: 'Courier New', Courier, monospace;">close()</span>, were added, so you can control output in between prints, and at the end.<br />
<br />
The two java files are available via Assembla, at <a href="http://www.assembla.com/code/hastur/subversion/nodes/trunk/Hastur_J/src/com/flyingspaniel/pdf">http://www.assembla.com/code/hastur/subversion/nodes/trunk/Hastur_J/src/com/flyingspaniel/pdf</a><br />
<br />
This code requires the wonderful <a href="http://itextpdf.com/">iText</a> library, either 2.1.7 or 5. Enjoy!Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-13142342201147590022012-06-15T12:47:00.000-07:002012-06-15T12:47:29.920-07:00How to Parallelize your Algorithm with an ExecutorService. Part 1, the "Happy Path"<div>
Let's say you have a long running algorithm to calculate stuff, for example, the median and 99th </div>
<div>
percentile (and other percentiles) of IRS income data. This is slow because calculating medians and percentiles typically requires sorting data, and there's a lot of IRS data. Your method looks like:</div>
<div>
<br />
<br /></div>
<pre class="brush:java"> public static PercentileResults calcPercentileResults (IRSData irsData, int year, Options options) {
BigObjectLotsOfData = irsData.getData(year, options);
// big calculation here
return percentileResults;
}
</pre>
You always need a year. Options represents optional complex things such as "only Married filing<br />
jointly", "only ages 55-65", etc. PercentileResults is a Business Object with the results. Currently, the code calls this in serial and puts results into a Map:<br />
<br />
<pre class="brush:java"> for (int year = 2000; year <= 2011; year++) {
PercentileResults result = calcMedianAnd99(irsData, year, options);
resultsMap.put(year, result);
}
</pre>
Obviously calculations for different years (and Options) can be done totally independently in parallel. An <a href="http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html">ExecutorService </a>helps manage this for you. Most of the ExecutorService methods want a <a href="http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Callable.html">Callable</a>, so the first step is to convert your class into a callable. Here's a first pass:<br />
<br />
<pre class="brush:java"> public class CallableCalculation implements Callable<PercentileResults> {
public final IRSData irsData;
public final int year;
public final Options options;
public CallableCalculation(IRSData irsData, int year, Options options) {
this.irsData = irsData;
this.year = year;
this.options = options;
}
@Override
public PercentileResults call() throws Exception {
BigObjectLotsOfData bolod = irsData.getData(year, options);
PercentileResults result = new PercentileResults();
// big calculation here that sets stuff in PercentileResults
bolod = null; // important to free this memory
// placeholder for Option 1 and 2 (see below)
return result;
}
}
</pre>
For simplicity, I made all the values <span style="font-family: 'Courier New', Courier, monospace;">public final</span> so that they could be accessed later. If you don't like this style, make them private and add accessors as desired. More on this later...
The simple way to call this in parallel would be:
<br />
<br />
<br />
<pre class="brush:java"> public Map<Integer, PercentileResults> calculateInParallel(Options options) throws Exception {
IRSData irsData = IRSData.getInstance();
HashMap<Integer, PercentileResults> resultMap = new HashMap<Integer, PercentileResults>();
ArrayList<CallableCalculation> tasks = new ArrayList<CallableCalculation>();
for (int year = 2000; year <= 2011; year++) {
CallableCalculation cc = new CallableCalculation(irsData, year, options);
tasks.add(cc);
}
int processors = Runtime.getRuntime().availableProcessors();
//might want to adjust that number some...
ExecutorService myService = Executors.newFixedThreadPool(processors);
// oops - there's a problem coming up
List<Future<PercentileResults>> results = myService.invokeAll(tasks);
for (Future<PercentileResults> future : results) {
PercentileResults pr = future.get();
resultMap.put(year, pr); // oops, what's the year for that Future???
}
return resultMap;
}
</pre>
<pre class="brush:java"></pre>
<br />
Now, there's one "gotcha" so far. The input parameter <span style="font-family: 'Courier New', Courier, monospace;"><b>year </b></span>has gotten separated from the results.<br />
There's a few options.<br />
<br />
1) If the final location is really really clear and obvious, the algorithm itself could put the results<br />
there. In this example, before returning the result (see comment "placeholder"), just put the results<br />
into the Map. This is a simple solution but not very robust to changes in requirements.<br />
<br />
2) Add the year as a new field to PercentileResults, and set it (at placeholder spot). This is robust.<br />
But tedious and violates <a href="http://c2.com/cgi/wiki?DontRepeatYourself">DRY</a>. What if there are <i>lots </i>of settings you want to remember? Like all the <br />
Options? And maybe you don't want to clutter your XXXResults with the input settings. Or you can't - it's taken from some third party library.<br />
<br />
3) The option I like best it to <i>return the CallableCalculation</i>! It already holds all the settings. You just need to add a field for the results, and relevant accessors. So you aren't doing much extra work. Your class would look like this (changes noted by "NEW")<br />
<br />
<br />
<pre class="brush:java">public class CallableCalculation implements Callable<CallableCalculation> {
public final IRSData irsData;
public final int year;
public final Options options;
PercentileResults results; // NEW
public CallableCalculation(IRSData irsData, int year, Options options) {
this.irsData = irsData;
this.year = year;
this.options = options;
}
public PercentileResults getPercentileResults () { return results; } // NEW
@Override
public CallableCalculation call() throws Exception {
BigObjectLotsOfData bolod = irsData.getData(year, options);
results = new PercentileResults();
// big calculation here that sets stuff in PercentileResults
bolod = null; // important to free this memory
return this; // NEW
}
}
</pre>
and the calling method, in place of the "oops - there's a problem coming up" section has:<br />
<br />
<br />
<pre class="brush:java">List<Future<CallableCalculation>> results = myService.invokeAll(tasks);
for (Future<CallableCalculation> future : results) {
CallableCalculation cc = future.get();
resultMap.put(cc.year, cc.getPercentileResults());
}
</pre>
In a future post we will consider the unhappy path with errors and Exceptions.Morgan Conradhttp://www.blogger.com/profile/15963409751532704983noreply@blogger.com1tag:blogger.com,1999:blog-6276165447167620036.post-26139171066712582812012-04-29T17:01:00.000-07:002012-04-29T17:08:36.355-07:00From java.awt.print.Pageable to PDFMy current project has implemented multi-page printing, with headers and footers and all that. It's complex. We use some of the ideas from <a href="http://java-sl.com/about_author.html">Stanislav Lapitsky</a> and his series of articles about a <a href="http://java-sl.com/Pagination_In_JEditorPane_Print.html">PaginationPrinter</a>. If a user wants the printouts <i>not on paper</i>, but as PDF files, we suggest that they use a free utility like <a href="http://www.cutepdf.com/">CutePDF</a>. (there are alternatives)<br />
<br />
However, CutePDF is Windows only, and it's a couple of extra steps as they have to remember to select the correct "printer", then enter a filename. Also, there may be a future need to append output from a collection of data into a single output PDF file.<br />
<br />
So, a cross-platform, java-centric way to "print" our results by creating a PDF file would be a good option. Fortunately, <a href="http://itextpdf.com/">iText </a>offers everything we need, in fact, way more than we need, and you can get an excellent book <a href="http://itextpdf.com/book/">iText in Action</a>.<br />
<br />
Some Googling for help was tricky, cause, if you Google for something like "java print PDF" you get lots of links on how to open and print an existing PDF document. We want to <i>create </i>a PDF document. More searching led to an excellent blogpost by Gert-Jan Schoeten, "<a href="http://www.zenbi.co.uk/2011/09/04/printable-to-pdf/">From java.awt.print.Printable to PDF</a>". You would not do too poorly by skipping the rest of my blog and just going to his. However, by using more information from Pageable and Printable you can slightly simplify his code. The way I set things up and return the results is, IMO, slightly preferable, but still largely a matter of style. I am largely glossing over the whole FontMapper issue as well, since my project uses only basic fonts.<br />
<br />
Here's the code:<br />
<br />
<br />
<pre class="brush:java">package com.flyingspaniel.pdf;
import java.awt.Graphics2D;
import java.awt.print.PageFormat;
import java.awt.print.Pageable;
import java.awt.print.Printable;
import java.awt.print.PrinterException;
import java.io.IOException;
import java.io.OutputStream;
// if you change to iText v5 these imports change to com.itextpdf...
import com.lowagie.text.Document;
import com.lowagie.text.DocumentException;
import com.lowagie.text.Rectangle;
import com.lowagie.text.pdf.DefaultFontMapper;
import com.lowagie.text.pdf.FontMapper;
import com.lowagie.text.pdf.PdfContentByte;
import com.lowagie.text.pdf.PdfWriter;
/**
* With this class, you can print a {@link Pageable} to a pdf
*
* @author Morgan Conrad
*
* Based upon earlier work by
* @author G.J. Schouten
* @see http://www.zenbi.co.uk/2011/09/04/printable-to-pdf/
*
* Using the wonderful iText library. This version is built for version 2.1.7 but
* it will work, with changes to the import statements, with iText 5.2.1
* @author Bruno Lowagie
* @see http://itextpdf.com/
*/
public class PDFPrinter {
protected final FontMapper fontMapper;
static DefaultFontMapper sDefaultFontMapper = null;
static String sFontDirectory = "C:/windows/fonts";
/**
* Constructor
* @param fontMapper if null, uses the DefaultFontMapper
*/
public PDFPrinter(FontMapper fontMapper) {
this.fontMapper = fontMapper != null ? fontMapper : getDefaultFontMapper(sFontDirectory);
}
/**
* Default constructor, uses the DefaultFontMapper
*/
public PDFPrinter() {
this(getDefaultFontMapper(sFontDirectory));
}
/**
* Change the directory for fonts. (OS dependent)
* Generally, one should call this *before* ever instantiating a PDFPrinter
*
* @param fontDirectory
*/
public static synchronized void setFontDirectory(String fontDirectory) {
sFontDirectory = fontDirectory;
sDefaultFontMapper = null; // force getDefaultFontMapper to recalculate...
}
private static synchronized FontMapper getDefaultFontMapper(String fontDirectory) {
if (sDefaultFontMapper == null) {
sDefaultFontMapper = new DefaultFontMapper();
if (sFontDirectory != null)
sDefaultFontMapper.insertDirectory(sFontDirectory);
}
return sDefaultFontMapper;
}
/**
* Creates a PDF from a Pageable
* @param pageable
* @param os
* @param closeStream whether to close the stream.
* Since we don't create the stream, best practice is to leave this false
* @return number of pages actually printed
*
* @throws IOException
* @throws PrinterException
*/
public int printToPdf(Pageable pageable, OutputStream os, boolean closeStream) throws IOException, PrinterException {
// sanity check
if (pageable.getNumberOfPages() == 0)
return 0;
// base page sizes on the first page
PageFormat pageFormat = pageable.getPageFormat(0);
float width = (float)pageFormat.getWidth();
float height = (float)pageFormat.getHeight();
Rectangle pageRect = new Rectangle(0.0f, 0.0f, width, height);
Document document = new Document(pageRect);
PdfWriter writer;
try {
writer = PdfWriter.getInstance(document, os);
} catch (DocumentException e) { // don't throw as an iText exception so other classes don't need links to iText classes
throw new RuntimeException(e);
}
writer.setCloseStream(closeStream);
document.open();
PdfContentByte contentByte = writer.getDirectContent();
int pageIdx = 0;
int pageStatus;
do {
if (pageIdx > 0)
document.newPage();
// The following is deprecated in iText5 but works
Graphics2D g2d = contentByte.createGraphics(width, height, fontMapper);
// if you are using iText5 and want to be "up to date", use
// Graphics2D g2d = new PdfGraphics2D(contentByte, width, height, fontMapper);
Printable printable = pageable.getPrintable(pageIdx);
try {
pageStatus = printable.print(g2d, pageFormat, pageIdx);
} finally {
g2d.dispose();// iText in Action book says this is very important, so put in a finally clause...
}
}
while ((pageStatus == Printable.PAGE_EXISTS) && (++pageIdx < pageable.getNumberOfPages()));
document.close();
writer.close();
os.flush();
return pageIdx;
}
}
</pre>
<br />
Hope this helps out some of you who wish an easy way to create PDF files from Swing components.<br />
<br />
Note that this code as written works in iText 2.1.7, but, with just changes to the imports it also works with iText 5.2.1. The version 5 is obviously more "up to date" but the licensing has changed. I don't blame Bruno Lowagie for trying to make some money from his great efforts. Here's a largely civil <a href="http://java.dzone.com/tips/java-pdf-tools-alternatives">discussion on the matter</a>.<br />
<br />
<br />Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com1tag:blogger.com,1999:blog-6276165447167620036.post-53988188688454294942012-01-18T16:33:00.000-08:002012-01-18T17:16:12.865-08:00A Brief Detour into JavaScript, XHTML, and HTML5We will return to Ubuntu and Hadoop later...<br />
<br />
My wife helps maintain a website for a volunteer organization, and I am powerless to help much because, despite years of programming, I have essentially zero knowledge of JavaScript and XHTML. Sure, I've read some JavaScript and XHTML, but all writing has been like my SQL "writing": take some existing code, make an extremely minor change, and hope for the best. What I saw of XHTML just felt "strange". Not as bad as early editions of J2EE, with <span style="font-family: 'Courier New', Courier, monospace;">Factories</span>, <span style="font-family: 'Courier New', Courier, monospace;">FactoryFactories</span>, and <span style="font-family: 'Courier New', Courier, monospace;">FactoryFactoryStrategies</span>, (o.k., I'm kidding some) but pretty bad.<br />
<br />
But, it would be useful to actually know a bit more. Also, I want to learn HTML5, so it seemed like a good time to learn some basics of JavaScript and XHTML. So, to get started, I visited the local library and checked out <i><a href="http://www.amazon.com/JavaScript-AJAX-Dummies-Andy-Harris/dp/0470417994/ref=sr_1_sc_1?ie=UTF8&qid=1326930504&sr=8-1-spell">JavaScript and AJAX for Dummies</a></i> by Andy Harris. I already own a couple of HTML5 books.<br />
<br />
From Andy's book, here is the XHTML version of the classic "Hello World" program. I have removed all the < and > cause they screw up blogger.<br />
<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">html lang="EN" dir="ltr" xmlns="http://www.w3.org/1999/xhtml"</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> head</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> meta http-equiv="content-type" content="text/xml; charset=utf-8" /</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> titleHelloWorld.html/title</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> script type = "text/javascript"</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> //![CDATA[</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> // Hello, world!</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> alert("Hello, World!");</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> //]]</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> /script</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> /head</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> body</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> /body</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">/html</span><br />
<br />
<br />
<pre></pre><div><span style="font-family: inherit;">Now, I'm a Java programmer and used to <i>some </i>verbosity, but this really pushed the limits. Speaking as a complete outsider from Mars looking at this, <b><i>what the Heck were they thinking</i></b>? There's the !DOCTYPE tag that I guess you just get used to typing, or copy from a pre-existing file. Same for the HTML lang="en" etc... </span></div><div><span style="font-family: inherit;"><br />
</span></div><div><span style="font-family: inherit;">There's the meta tag. What's that even mean? That I'm UTF-8 and text/xml. I know about internationalization, where it might not be UTF-8. But I already said that I'm English via lang="EN". And UTF-8 is the natural encoding for English, should be the default, shouldn't it? Again, <b>what were they thinking</b>? </span></div><div><span style="font-family: inherit;"><br />
</span></div><div><span style="font-family: inherit;">Finally, the really mysterious //CDATA stuff. Now, CDATA means "Character data". If the XML wizards are going to force me to write 174 characters (for the DOCTYPE and html tags) just to get started, why are they suddenly trying to save a few characters here??? Just call it CHARACTER_DATA or whatever. Secondly, why is this commented out??? I think I finally understand, it's commented out with "//" so that JavaScript will ignore it. And, fortunately, XML parsers do not use "//" for comments so they will not ignore it. What a hack! Now, I'd probably appreciate the hack in some deep part of the Java Runtime or the Linux kernel. But as something out in the open that everybody is supposed to use around every JavaScript, all I can say, yet again, is <b>what were they thinking</b>? It's quite obvious that whoever wrote the XHTML standard had no thought for usability, elegance, or thought. Whenever they saw a hoop for coders to jump through, they added two just to make sure. I expect someday to see it revealed that XHTML was actually a social experiment like the <a href="http://www.prisonexp.org/">Stanford Prison Experiment</a> - given a big-shot committee, just how much pain and agony will users tolerate?</span></div><div><span style="font-family: inherit;"><br />
</span></div><div><span style="font-family: inherit;">By contrast, here is the HTML5 version. (Again, < and > removed)</span></div><div><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><br />
</span></div><div><div><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"></span></span></div><div><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"></span></span></div><div><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">!DOCTYPE HTML</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">html</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> head</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> titleHello World/title</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small; white-space: pre;"> </span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> script type="text/javascript"</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> alert("Hello World");</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> /script</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> /head</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> body</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> /body</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">/html<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span></span></div><div><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span></span></div><div><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"></span></span></div></div><div><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><br />
</span></div><div><span style="font-family: inherit;">Notice something? All that crap is gone! Wonderful.</span></div><div><span style="font-family: inherit;"><br />
</span></div><div><span style="font-family: inherit;">As it turns out, my wife might want a Calendar thingamabob on her page. You can find lots of JavaScript code to do this. I found some code, a <i>mere </i>220 lines long (sarcasm intended), that you can cut and paste into your page. With HTML5, you can do it with 0 lines of code: just say input type="date".</span></div><div><span style="font-family: inherit;"><br />
</span></div><div><span style="font-family: inherit;">In conclusion, I am exceeding glad that I never bothered to learn XHTML. I hope to remain, as much as possible, proudly ignorant of XHTML. HTML5 has made an instant convert.</span></div><div><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><br />
</span></span></div>Morgan Conradhttp://www.blogger.com/profile/15963409751532704983noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-82052768641876066492012-01-03T08:34:00.000-08:002012-01-03T09:47:53.737-08:00Adventures in Ubuntu and Hadoop Part 4Let's recap Rocky and Bullwinkle's adventures with Hadoop and Ubuntu on an old desktop computer. They fairly easily <a href="http://flyingspaniel.blogspot.com/2011/12/adventures-in-ubuntu-and-hadoop-part-1.html">installed Ubuntu, Java and Eclipse</a>, then <a href="http://flyingspaniel.blogspot.com/2011/12/adventures-in-ubuntu-and-hadoop-part-2.html">installed an SSH server</a> and set the ip4 address, and then, in their greatest adventure, <a href="http://flyingspaniel.blogspot.com/2011/12/adventures-in-ubuntu-and-hadoop-part-3.html">finally got X-Windows (and VNC)</a> working. Now we are finally ready to install Hadoop. I plan to follow the <a href="http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/">blog post by Michael Noll</a> and the <a href="http://www.amazon.com/Hadoop-Definitive-Guide-Tom-White/dp/0596521979">O'Reilley Hadoop book</a> by Tom White (esp. Appendix A). According to Michael Noll, I have more that satisfied the prereqs.<br /><br />Recent Hadoop releases are <a href="http://www.takeyellow.com/apachemirror//hadoop/core/">here</a> or <a href="http://apache.deathculture.net/hadoop/core/">here</a>. Some strange domain names. As of today 0.20.203 was the latest stable release, so I grabbed it and put it in /opt/hadoop. Then, per Michael's instructions<br /><br /><span style="font-weight: bold;"> </span><span style=" font-weight: bold;font-family:courier new;" >sudo tar xzf hadoop-0.20.203.0rc1.tar.gz</span><span style="font-weight: bold;"> and</span> <span style="font-weight: bold;"> </span><span style=" font-weight: bold;font-family:courier new;" >sudo chown -R hduser:hadoop hadoop-0.20.203.0</span><br /><br />Then edit <span style="font-family:courier new;">/home/hduser/.bashrc</span> (don't forget to type sudo!) to add the following lines at the end. YMMV depending on exactly where your Hadoop and Java are installed.<br /><br /><br /><span style="font-family: courier new;font-family:courier new;" ># Set Hadoop-related environment variables</span><span style="font-family:courier new;"> </span><span style="font-family: courier new;font-family:courier new;" ><br />export HADOOP_HOME=/opt/hadoop/hadoop-0.20.203.0</span><span style="font-family:courier new;"> </span><br /><br /><span style="font-family: courier new;font-family:courier new;" ># Set JAVA_HOME<br /></span><span style="font-family: courier new;font-family:courier new;" >export JAVA_HOME=/opt/java/32/jdk1.6.0_30</span><span style="font-family:courier new;"><br /></span><span style="font-family: courier new;font-family:courier new;" ><br /># Add Hadoop bin/ directory to PATH</span><span style="font-family:courier new;"><br /></span><span style="font-family: courier new;font-family:courier new;" >export PATH=$PATH:$HADOOP_HOME/bin</span><br /><br />The Hadoop book suggests that you test if it will run by typing <span style=" font-weight: bold;font-family:courier new;" >hadoop version</span>. Before this will work,either re-login to run the .bashrc script, or manually do all <span style="font-style: italic;">three</span> exports. If you forget to export JAVA_HOME, you'll see a useful, informative message<br /><br /><span style="font-family:courier new;">Error: JAVA_HOME is not set.</span><br /><br />But, once you set all three, you'll see something like<br /><br /><span style="font-family:courier new;">mpc@mpc-desktop:/home/hduser$ <span style="font-weight: bold;">hadoop version</span></span> <span style="font-family:courier new;"><br />Hadoop 0.20.203.0</span> <span style="font-family:courier new;"><br />Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203 -r 1099333</span> <span style="font-family:courier new;"><br />Compiled by oom on Wed May 4 07:57:50 PDT 2011</span><br /><br />Wohoo! Our work is done! Well, not really, there's still a whole bunch to go, like configuring the Hadoop Distributed File System. (HDFS). But, let's declare victory for now and return to that on a later day.Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com0tag:blogger.com,1999:blog-6276165447167620036.post-70679337385949424562011-12-31T12:08:00.000-08:002012-01-03T08:34:15.363-08:00Adventures in Ubuntu and Hadoop Part 3<span style="font-family:georgia;font-size:100%;">Wherein Rocky and Bullwinkle install PuTTY and XMing to allow X-Windows access to the Ubuntu machine, or, "Maybe I'll use VNC instead".</span><div style="font-family:georgia;"><span style="font-size:100%;"><br /></span></div><div style="font-family:georgia;"><span style="font-size:100%;"><span style="font-size:100%;">When I used Unix a lot (20+ years ago) I was using X-Windows. I remember when <a href="http://en.wikipedia.org/wiki/Twm">Tom's Window Manager (twm)</a> was hot stuff. So I wanted to run an X "server" (poor choice of terms but that's what they call it) on my Windows laptop. The first step is to download and install <a href="http://www.chiark.greenend.org.uk/%7Esgtatham/putty/"></a><a href="http://www.chiark.greenend.org.uk/%7Esgtatham/putty/"><span style="text-decoration: underline;">PuTTY</span></a>, a highly-regarded telnet and SSH client. Be sure to get the latest release version 0.61 or higher if you are using Windows 7. PuTTY is pretty simple to use: double click on putty.exe, you'll see a (somewhat complex) dialog for the configuration. And you'll notice that on-line documentation it a bit inconsistent on how to configure it for XWindows. Mainly on the Connection-SSH-X11 page. I left it blank, as <a href="http://www.blogger.com/www.no.embnet.org/guides/puttyandxming.pdf">many</a> (<a href="http://the.earth.li/%7Esgtatham/putty/0.62/htmldoc/Chapter3.html#using-x-forwarding">also see PuTTY docs</a>) suggest.</span><br /></span></div><div><br /></div><div><br /></div><div><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 320px; height: 308px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgsrvvDz2QBDjkw79Cds4OIJpinUBWOKP7Glrami2yrtO74es5s4o23fY0kO8SDOoKaziTl8fwVsEKrnbloUxi4R9jLNMELa5pN6X4lp_0x6SEIq7jNszkDHz9buPru2cvVZKKAMczV4TE/s320/Putty.gif" alt="" id="BLOGGER_PHOTO_ID_5692389777471627730" border="0" /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><span style="font-size:100%;">All in all, PuTTY is a great little program.<br /></span><div><pre style="font-family:georgia;"><span style="text-align: left; white-space: normal; font-family:georgia;font-size:100%;" >XMing is more confusing. <a href="http://www.straightrunning.com/XmingNotes/">Installing</a> offers three versions, ("Website Release", "Public Domain", and "Work in progress" of multiple programs, "Xming", </span><span style="text-align: left; white-space: normal; font-family:georgia;font-size:100%;" >"XMing-portablePuTTY", "Xming-fonts", "Xming-mesa", and "Snapshot Xming". Once you figure out that the author, quite reasonably, wants a donation for the newer "Website Release" versions, you'll download the "Public Domain" versions for initial testing. I think that Xming-mesa is only for high-performance OpenGL type stuff, so I skipped that.<br /><br /></span><span style="text-align: left; font-size:100%;" ><span style="font-family:georgia;font-size:100%;"><span style="white-space: normal; ">Unlike the remarkably simple PuTTY, You'll see </span></span><span style="font-family:georgia;font-size:100%;"><i style="white-space: normal;">a lot</i></span><span style="white-space: normal; "><span style="font-family:georgia;font-size:100%;"> of .exe files in the Xming folder. Unlike PuTTY, where double-clicking always brings up a dialog to set prefs and show you that something is happening, Xming has a two step process. First the XLaunch.exe launcher, used to set prefs, then a separate Xming.exe which actually runs the X Server. Double clicking on Xming does not bring up a dialog. Instead, you should see an X icon in the Windows tool bar to show that it is running successfully. But, at least at first, you really want to run XLaunch first to set prefs. On the first page, there is, once more, a somewhat mysterious "Display number" entry. The examples give it a number but the docs suggest that leaving it blank may be better. However, in my version, I couldn't leave it blank. So I used 0.</span><br /></span></span></pre><pre style="text-align: -webkit-auto; "><span style="text-align: left; font-family: georgia;font-family:georgia;font-size:100%;" ><span style="white-space: normal; ">Xming comes with plink.exe, which is some sort of command-line version of PuTTY with the world's worst user interface. Double click on it, and you'll see a large dialog panel PuTTY Link:Usage with all the options. Seems like a good idea. Until you try to use it. Let's say I see a hard-to-type option I want to use, like -pgpfp. You <i>can't copy and paste. What's worse, you can't even read the option and type it, cause the freaking dialog is modal! </i>You have to memorize (or write down) the options, then close the dialog. Somebody went to extra work to create this dialog which is far less useful than the simple</span></span><span style="white-space: normal; text-align: left; font-family: georgia;font-family:Georgia, serif;font-size:100%;" ><span style="font-family: georgia;font-family:georgia;font-size:100%;" >, </span><span style="font-family:georgia;font-size:100%;"><span style="font-size:100%;"><span style="font-family:georgia;">time-tested "just print it to stdout" style.</span><br /></span><br /><span style="font-size:100%;">So, ignoring plink, let's continue. Run XLaunch and click through Next on every page. Save if you want. Then launch Xming and verify that you see an X icon in the task bar. You can right click and view the log file. Do so - it's useful till you get going. Then launch PuTTY, entering the IP address of the Linux box and setting up X11 forwarding. You should see a terminal window with the IP address of the Linux box asking you to login. Do so.<br /></span><br /><span style="font-family:georgia;">Try to start an X-Window command, like xclock &. You'll get an error message along the lines of "Can't open Display". Now, this is a remarkably poor and confusing error message. What is really should say is something like Can't open $DISPLAY "", with the quotes around your $DISPLAY variable. That's because your $DISPLAY variable isn't set. You will need the</span><a style="font-family: georgia;" href="http://www.littletechtips.com/2009/10/connecting-to-xming-from-remote-system.html"> "magic incantation"</a></span><span style="font-size:100%;"><br /><br /></span></span><span style="font-family:georgia;font-size:100%;"><span style="font-family:courier new;">export DISPLAY=windowsmachineipaddress:0.0</span><br /><br /></span><span style="text-align: left;font-size:100%;" ><span style="font-family:Georgia, serif;"><span style="white-space: normal;"><span style="font-size:100%;"><span style="font-family:georgia;">Where you enter the IP address of the Windows box. If, like many people, you are using a router with DHCP, you've got to startup a command prompt on the Windows box and type ipconfig -all to get your IP address. Annoying.</span><br /><br /><span style="font-family:georgia;">Alas, it still doesn't quite work. If you look at the XMing log, you'll see something like:</span><br /></span><br /></span></span></span><span style="font-size:100%;"><span style="font-size:130%;">Xming: client 4 rejected from IP ipaddress_of_archlinuxbox}<br /></span><br /><br /></span><span style="text-align: left; font-size:130%;" ><span style="font-family:Georgia, serif;"><span style="white-space: normal; "><a href="http://www.javamonamour.org/2009/12/xming-in-log-i-find-client-4-rejected.html">As noted here</a>, the fix is to go to the Xming shortcut and add -ac at the end of the command line, or to add the IP address of the Linux box to the X0.hosts file. Which may be tricky right now cause the Xming process has the file open. I added the -ac option and finally, finally, I can get an xclock and xeyes on the Windows display. Victory!<br /></span></span></span></pre><pre style="text-align: -webkit-auto; "><span style="text-align: left; font-size:100%;" ><span style="font-family:Georgia, serif;"><span style="white-space: normal; "><br /></span></span></span></pre><pre style="text-align: -webkit-auto; "><span style="text-align: left; font-size:130%;" ><span style="font-family:Georgia, serif;"><span style="white-space: normal; "><span style="font-weight: bold; font-family:georgia;" >By contrast, VNC is a piece of cake.</span><span style="font-family:georgia;"> I used </span><a style="font-family: georgia;" href="http://www.tightvnc.com/">TightVNC</a><span style="font-family:georgia;">. Installing is simple. For my purposes, you definitely need to install the viewer. The server (from your Windows machine) is optional.</span><br /><br /><span style="font-family:georgia;">To use the viewer, run it, type in the address of the server, and it pretty much just works. The main thing I changed is under the Options... dialog, where I set Local Cursor Shape to "Normal Arrow". The dot cursor was too small for my taste, YMMV.</span><br /></span></span></span></pre><pre style="text-align: -webkit-auto; "><span style="text-align: left; "><span style="font-size:100%;"><span style="font-family:Georgia, serif;"><span style="white-space: normal; "><br /></span></span></span></span></pre><pre style="text-align: -webkit-auto; "><span style="text-align: left; "><span style="font-size:100%;"><span style="font-family:Georgia, serif;"><span style="white-space: normal; "><br /></span></span></span></span></pre></div>Morgan Conradhttp://www.blogger.com/profile/08488994506896975339noreply@blogger.com1