Monday, November 8, 2010

Scala & Eclipse...Just keep cleaning...

Today's aggravation is most likely brought to you by the Scala IDE (v1.0.0.201009232358) plugin for Eclipse (v3.5.2.R35).  A simplified version of the offending code is:
  def nestedDoesntWork(num: Int) = {
    if (true)
      for (x <- 0 until num)
        println(x)
  }

Where I have some condition that is checked and if true, proceeds into a for-loop.  The error that I repeatedly get is:
  • Exception in thread "main" java.lang.NoSuchMethodError: scala.collection.immutable.Range$ByOne.foreach$mVc$sp(Lscala/Function1;)V
        at org.spin.node.gogrid.Client$.doesntWork(Client.scala:62)
        at org.spin.node.gogrid.Client$.main(Client.scala:72)
        at org.spin.node.gogrid.Client.main(Client.scala)
However, this seems to work:
  def nestedWorks(num: Int) = {
    if (true)
      for (x <- (0 until num).toList)
        println(x)
  }

Very odd. Even odder is if I define either of these functions on the Scala interpreter in Eclipse, they work fine. It's only when the nestedDoesntWork is defined within a class and I try to run a main function calling that class. The same happens if I take out the if(true) line.

Solution? Clean project...and look, everything is fine and dandy. Make a change, get the error? Clean the project, and hurray.  Note that the nestedWorks always works...without cleaning.

Annoying to say the least, but today's lesson is if it doesn't work when it should the first time, clean the project.  If it still doesn't work, clean it again. and again. and again.

Monday, November 1, 2010

Scala & Json (Lift-web): Trouble with "mapping" field name to another name

This is a trivial mistake that just ate up the last 2 hours of my life that I will never get back again.

One thing that is important is to be able to take the "raw" parsed json object and rename some of the fields since GoGrid uses some names that are reserved and/or have other Scala meanings.  For example, GoGrid uses Object and Option to mean things specific to their environment.  

This is possible by using the JValue's map function to postprocess AST.  Following the example in the readme documenation, I reproduced the following:
package org.gogrid.sandbox
abstract class jsonSandbox
{ }

case class Person(firstname: String)

And then started up the Scala interpreter:
Welcome to Scala version 2.8.0.final (Java HotSpot(TM) Server VM, Java 1.6.0_20).
Type in expressions to have them evaluated.
Type :help for more information.

scala> import org.gogrid.sandbox._

scala> import net.liftweb.json.JsonParser._
import net.liftweb.json.JsonParser._

scala> implicit val formats = net.liftweb.json.DefaultFormats 
formats: net.liftweb.json.DefaultFormats.type = net.liftweb.json.DefaultFormats$@1c9fe7e

scala> val jsonString = """ { "first-name" : "Jim" } """
jsonString: java.lang.String =  { "first-name" : "Jim" } 

scala> val json = parse(jsonString)
json: net.liftweb.json.JsonAST.JValue = JObject(List(JField(first-name,JString(Jim))))

scala> json map {
     | case JField("first-name", x) => JField("firstname", x)
     | case x => x
     | }
<console>:16: error: not found: value JField
       case JField("first-name", x) => JField("firstname", x)
            ^

Heh? Why is JField not found???

My "Oh DUH moment":

I did NOT import the package where things like JField, JArray are defined.
 
Insert wall bash here.
scala> import net.liftweb.json.JsonAST._
import net.liftweb.json.JsonAST._

scala> json map {
     | case JField("first-name", x) => JField("firstname", x)
     | case x => x
     | }
res4: net.liftweb.json.JsonAST.JValue = JObject(List(JField(firstname,JString(Jim))))

Dang it.

Scala & Json (Lift-web): Trouble with getting started

I'm abandoning using XML parsing for this project.  While I can get the results back from GoGrid now, I can't seem to figure out a simple, clean way to get the result string transformed into the objects for manipulation in our code.  GoGrid has the XML set up as basically key value pairs, where for each object 'type' is specified as an attribute to the element, which makes squishing the result string through something to pop out an object at the end kind of an ugly hassle.

GoGrid does provide the default response format to be Json, so I'm now chasing down this avenue.

First up is to find an appropriate library to handle the heavy lifting, and for this I'm using Lift-Web's Json library.  Lift-web is a large project made up of several projects, for my purposes, I only need the Json module. It took a little bit of scrounging around the net, but I found that what I need is the lift-json module.  The getting started documentation that I found on lift-web's site tells you how to set up a project from scratch, which is not what I need.  I just needed access to this particular lib.  After browsing their source tree, I decided to just put the information into my project's pom that I found here combined with the announcement that Lift-Web 2.1 has been released and inserted the following into my pom:
//DOES NOT WORK

 net.liftweb
 lift-json
 2.1

//DOES NOT WORK
This (as the comments note) does not work.  I ended up with the following error when I tried to save the pom:

Missing artifact net.liftweb:lift-json:jar:2.1:compile

Okay, so that didn't work.  So then I tried using version 2.0 instead to see if that will work:
//SORT OF WORKS

    net.liftweb
    lift-json
    2.0

//SORT OF WORKS

Well this seemed promising.  I was able to save the pom with no errors.  So the next step was to give the JsonParse -ing a whirl in the command line Scala interpreter (Eclipse v3.5.2 with Scala IDE for Eclipse v1.0.0.201009232358 plugin).  I found an example on Stackoverflow and tried it out for myself:
scala> import net.liftweb.util.JSONParser
:8: error: value util is not a member of package net.liftweb
       import net.liftweb.util.JSONParser

Um, okay. So that didn't work.  Looking through the library that I added, I decided to try this next:
scala> import net.liftweb.json
import net.liftweb.json

scala> val jStr = """ { "name" : "Jim Bob" } """
jStr: java.lang.String =  { "name" : "Jim Bob" } 

scala> json.JsonParser.parse(jStr)
error: error while loading JsonParser, Scala signature JsonParser has wrong version
 expected: 5.0
 found: 4.1 in /home/sophia/.m2/repository/net/liftweb/lift-json/2.0/lift-json-2.0.jar(net/liftweb/json/JsonParser.class)
Heh?! What the heck is going on? After some more fruitless searching on the web, I looked back at this site and saw that there was also a lift-json_2.8.0 package. I am using Scala 2.8.0 ... so I changed my pom to:
//SUCCESS!!

 net.liftweb
 lift-json_2.8.0
 2.1

//SUCCESS!!
And... Hurray!  That did the trick:
scala> import net.liftweb.json
import net.liftweb.json

scala> val jStr = """ { "name" : "Jim Bob" } """
jStr: java.lang.String =  { "name" : "Jim Bob" } 

scala> json.JsonParser.parse(jStr)
res5: net.liftweb.json.JsonAST.JValue = JObject(List(JField(name,JString(Jim Bob))))

Another note for getting started is that the string that is being parsed can be escaped or not, they both work:
//Un-escaped quotes in json string
scala> val jStr = """ { "name" : "Jim Bob" } """
jStr: java.lang.String =  { "name" : "Jim Bob" } 

scala> json.JsonParser.parse(jStr)
res5: net.liftweb.json.JsonAST.JValue = JObject(List(JField(name,JString(Jim Bob))))

//Escaped quotes in json string
scala> val badJstr = " { \"name\" : \"Jim Bob\" } "
badJstr: java.lang.String =  { "name" : "Jim Bob" } 

scala> json.JsonParser.parse(badJstr)
res6: net.liftweb.json.JsonAST.JValue = JObject(List(JField(name,JString(Jim Bob))))
So the next thing on the list is to take a Json string, parse it and shove it into an object.  An example of this can be found on lift-web's source code site and scroll down to extraction. I decided to go for a very simple test, using just a list of numbers (I'm still on the console in Eclipse with Scala interpreter):
scala> import net.liftweb.json.JsonParser._
import net.liftweb.json.JsonParser._

scala> implicit val formats = net.liftweb.json.DefaultFormats 
formats: net.liftweb.json.DefaultFormats.type = net.liftweb.json.DefaultFormats$@7db754

scala> case class Winn(numbers : List[Int])
defined class Winn

scala> val jsonStr = """{ "numbers" : [1,2,3,4] } """
jsonStr: java.lang.String = { "numbers" : [1,2,3,4] } 

scala> val json = parse(jsonStr)
json: net.liftweb.json.JsonAST.JValue = JObject(List(JField(numbers,JArray(List(JInt(1), JInt(2), JInt(3), JInt(4))))))

scala> json.extract[Winn]
net.liftweb.json.MappingException: Parsed JSON values do not match with class constructor
args=
arg types=
constructor=public Winn(scala.collection.immutable.List)
 at net.liftweb.json.Meta$.fail(Meta.scala:128)
 at net.liftweb.json.Extraction$.instantiate$1(Extraction.scala:200)
 at net.liftweb.json.Extraction$.newInstance$1(Extraction.scala:222)
 at net.liftweb.json.Extraction$.build$1(Extraction.scala:240)
 at net.liftweb.json.Extraction$.extract(Extraction.scala:284)
 at net.liftweb.json.Extraction$.extract0(Extraction.scala:172)
 at net.liftweb.json.Extraction$.extract(Extraction.scala:40)
 at net.liftweb.json.JsonAST$JValue.extract(JsonAST.scala:288)
 at .(:17)
 at .()
 at RequestResult$.(:9)
 at RequestResult$.()
 at...

So close...but not quite there.  Back to the googling board.  I eventually found this in lift-web's markmail and they say that:

"The extraction feature uses paranamer to reflectively read the constructor parameter names. Unfortunately paranamer lib does not work with Scala console. Your example should work if you put your case class definitions into .scala file and then import those after compilation."

Okay, so now I created a new scala file in my project that contains the following case class:
package org.gogrid.sandbox
abstract class jsonSandbox
{ }

case class Winn(numbers : List[Int])

And then stopped the previous Scala interpreter and then right clicked over the package containing my sandbox class in the package explorer and started up a new Scala interpreter from there:
Welcome to Scala version 2.8.0.final (Java HotSpot(TM) Server VM, Java 1.6.0_20).
Type in expressions to have them evaluated.
Type :help for more information.

import org.gogrid.sandbox._

scala>
scala> Winn
res0: org.gogrid.sandbox.Winn.type = &ltfunction1>

scala> import net.liftweb.json.JsonParser._
import net.liftweb.json.JsonParser._

scala> implicit val formats = net.liftweb.json.DefaultFormats 
formats: net.liftweb.json.DefaultFormats.type = net.liftweb.json.DefaultFormats$@166de01

scala> val jsonStr = """{ "numbers" : [1,2,3,4] } """
jsonStr: java.lang.String = { "numbers" : [1,2,3,4] } 

scala> val json = parse(jsonStr)
json: net.liftweb.json.JsonAST.JValue = JObject(List(JField(numbers,JArray(List(JInt(1), JInt(2), JInt(3), JInt(4))))))

scala> val w = json.extract[Winn]
w: org.spin.node.gogrid.objects.Winn = Winn(List(1, 2, 3, 4))

scala> w.numbers
res3: List[Int] = List(1, 2, 3, 4)

scala> w.numbers(0)
res4: Int = 1

That's the best "Int = 1" I've seen in a long time :) .

So to summarize, to get started:
  1. Add a dependency on lift-json_2.8.0, version 2.1
    
     net.liftweb
     lift-json_2.8.0
     2.1
    
    
  2. You can choose to escape or not the quotes within the json string, they both work.
    scala> val jStr = """ { "name" : "Jim Bob" } """
    jStr: java.lang.String =  { "name" : "Jim Bob" } 
    
    scala> val badJstr = " { \"name\" : \"Jim Bob\" } "
    badJstr: java.lang.String =  { "name" : "Jim Bob" } 
    
  3. Case classes that represent the objects you are munging the string into cannot be defined in the class you are using them (and also can not be defined in the Scala interpreter of Eclipse).
  4. Must include the implicit formats line of code
    implicit val formats = net.liftweb.json.DefaultFormats 

    Otherwise you will get this error:
    error: could not find implicit value for parameter formats: net.liftweb.json.Formats