Confused? Type help



> blog post A Common Lisp web startup test-case, two years after the Reddit switch

A Common Lisp web startup test-case, two years after the Reddit switch
Tue, 18 Dec 2007 13:29:26

Memories, memories... Just a little bit longer than two years ago I had a plan to show that Common Lisp web programming wasn't such a disaster as the Reddit guys made it out to be. Never would I have imagined I'd be a paid Common Lisp web programmer working on a social networking site myself, two years later.

I thought it might be nice to offset the Reddit experience against the things we at (plug) stix.to are doing. In what way did we feel the problems they were mentioning?

Reddit summary

For you not in the know about this Reddit-CL horse which was beaten to death: In December 2005, Reddit (a vote up/down articles site a la Digg) switched from Common Lisp to Python. It created an uproar amongst people that like uproars, and Common Lisp was called all kinds of names.

Not so much by the Reddit people though. I've read up a bit on their Common Lisp experiences, but the Reddit folks didn't actually publish that much information about their CL setup and experiences. I mailed them about it but as for now no response. But what DO we know.

The biggest source of information I could find is their 'on lisp' blog-post:

  1. "One of the biggest issues was the lack of widely used and tested libraries."

  2. "Because of the low-level socket and threading code we had to write, Reddit would not run on my Mac, and I was always tethered to our FreeBSD development server. Not being able to program offline is a pain." We could generalize this as cross platform problems.

    That's about it actually. I also scraped some suggested and/or second hand information from the internet:

  3. Lack of CL coders

  4. Acquisition requisite, in general outside pressure

This list is quite arbitrary and perhaps not fair as far as Reddit is concerned, but I feel it gives a nice selection of issues which people have been concerned with concerning CL.

Stix.to introduction

Stix.to is also a social site of sorts. At the core you can put post-it notes, chats, voices, pictures, videos... on any page on the internet. And those you can share them with friends and groups. In the case of chats, you can communicate with whoever is on the same page and has permission to see them.

The entire codebase of stix.to is written in Common Lisp. That is to say, html output is written in s-expressions and all our Javascript is written in Parenscript (if we choose to ignore a few quick hacks). This might be standard fare nowadays in CL circles, but nevertheless I can not emphasize enough the joy I feel editing Javascript and HTML with s-expressions. Using an s-expression aware editor it is so much easier writing, editing and traversing those s-expressions then their native representation.

Our most important libraries: we use UnCommon Web as web framework, Elephant for persistence, the in-house written cl-json for communication between client and server and, as mentioned earlier, Parenscript.

Walking the list

So how does the previous listed list of issues square up against what we have experienced here at stix.to?


1. lack of mature libraries

I would actually say that this is our major problem. That is to say, our biggest annoyance is in a few packages that need considerable tinkering, some of which would surely be available in your favorite popular language X. It's for example quite probable that your popular language X has got a JSON handling library, and we had to build it from scratch.

But then there are libraries like Elephant, which implements persistent CL objects. We worked also quite a bit to improve Elephant, by speeding it up in some places and by adding a Postgresql backend. Agreed this costs time and money, but it gives us an exceptionally flexible persistence tool, hard to implement in most other languages.

The same could be said for UCW. Not that we pushed the library forward so much, but UCW's development model doesn't put stability on the first place. But what you get back is quite a powerful web toolset, with an unusually cool set of features.

So for us, the only libraries that gave us trouble were the ones giving us special abilities. In general, we hardly needed things that didn't function adequately, and the things missing were easily added.


2. cross platform problems

I still wonder what special socket and threading coding magic was needed for what is in essence quite a straight forward site. But what do I know. We at stix.to actually also have the same kind of setup, but thanks to virtualization software nowadays this has become a non-issue.

Two third of us who code are using MacBooks but develop our service in Linux virtual machines. When we started using the macs we used the beta version of VMware Fusion (VMware for the Mac). So we don't have to run into the kind of problem mentioned above.

As a plus we got some extra goodies: Linux, Windows and Mac compatibility checking with just one laptop is really nice. Also we're planning to integrate VMware (or similar), Selenium, and Darcs to have an automated test in clean OS environments. Virtualization is so cool.


3. lack of CL coders

This one I never understood. I heard it a number of times and I refuse to believe that this is an actual problem. We recently got a guy to work for us who hacks the socks of your standard programmer, and we found him with no problem at all. He practically drifted in view the day we started talking about expanding the workforce.

I'd think it's the other way around. How many people can and would love to code Common Lisp; for money? And how many jobs are actually offered?

Or do people have actual experience with trying to attract Common Lispers, and failed to do so. I'd love to hear stories about that.

And if you can't find them, educate them. I talked to guys a Streamtech who learned CL at the job with seemingly little adaption problems. I read the same kind of thing about ITA.


4. outside pressure

The only info i have on outside pressure is through talks with (potential) investors. And they don't exert pressure at all. And I find it fascinating to see how little investors are interested in the technology behind the service.

Except this one investor who nodded both knowingly and deeply sceptical when we mentioned the name Common Lisp. After some inquiry we found out he took a Lisp course at university when he was younger. At some institute of computer science. Beard style. He told that people around him called it the institution of extinct computer languages.

Mostly though, the only thing they're interested in is how easy your service is to copy. And in that way our use of Common Lisp is actually a plus. They just give you a free pass to rave about all the advance over the top features Common Lisp provides. After which you pretty much have to shut up about programming, to not come across as a real-world-shunning nerd.

But anyway this playing field is nothing like those big company cultures I'm hearing so much about. As a startup anyway, there doesn't seem to be anything to be worried about. And once you get to the Reddit-like buyout level, to me this would seem more like a luxury problem.


As of other problems; I don't really see them. Perhaps I'm just blind. Numb through swimming in a morphine-saturated pool. Now go on and start your Common Lisp powered companies! After the first dip, the water is nice and warm; nothing to fear...


comments

on Tue, 18 Dec 2007 16:44:55 Eric said:

I honestly just wanted to see if I got the captcha right..


on Tue, 18 Dec 2007 17:22:17 ?? said:

...


on Tue, 18 Dec 2007 17:38:24 Yo Soy said:

seeing if captcha can handle decmial points nicely (apparently not even single digit decimals!) sad.


on Tue, 18 Dec 2007 17:58:50 tieze said:

I know Yo Soy, I tried though, for days on end. Racked my brain. Typed till my fingers bled. But at least I've learned something: I'm not the coder I thought I was.

I know my limits now. And feel peace inside.

Not all coders can say that!


on Tue, 18 Dec 2007 18:11:49 w-g said:

Regarding the libraries problem: please help fix them! :-)

A nice thing would be some small investment in documenting, fixing bugs and making existing libraries more robust.

And good luck! :-)


on Tue, 18 Dec 2007 19:57:11 NIL said:

NIL


on Tue, 18 Dec 2007 20:24:23 heh said:

i like the captcha


on Tue, 18 Dec 2007 22:27:55 anonymoustroll said:

test


on Wed, 19 Dec 2007 00:02:28 Jim Bob said:

[2, 797, 179]

With the above string of seemingly meaningless digits, Jimbo joined two groups simultaneously:

1) the elite class of posters who defeated the CAPTCHA in its most difficult form 2) the group of people who leave spammy contents


on Wed, 19 Dec 2007 02:17:13 art said:

The problem with retaining CL (and Haskell, Erlang, etc.) developers is almost the exact opposite of retaining Java/C /Perl/etc. developers. You only have to talk to a couple professional Lisp developers to find one who can do the job, but the onus is on you to attract them.

Whereas, with Java developers, you have to go through 100 resumes, 30 phone screens, and 10 interviews to find three worth hiring, but one of those three will do any job for just about any amount of money you want to offer.

@captcha: FindInstance[a b c==361, {a,b,c}, Primes] kept me from having to think.


on Wed, 19 Dec 2007 03:51:50 Kyle Ambroff said:

I'm a .NET and GTK developer, and I would love to get payed for hacking on Common Lisp code. I'm sure you are right that there are more people who feel my pain.


on Wed, 19 Dec 2007 05:02:10 Tzury Bar Yocha said:

Reddit was re-written 3 times already, Common-Lisp, Web.py, and a brand new in-house python framework. Re writing 3 times the same application (without adding any significant features which are hard to impossible to implement with the current framework) talks for itself. If someone love common lisp no one can talk him/her out of it. That is Common Lisp it a true love. If you give up Common Lisp, say, you did not truly love it at the first place.


on Wed, 19 Dec 2007 05:21:18 Andrew said:

I currently hack all sorts of web apps in the usual languages, and would dearly love to get paid for writing Common Lisp. Ahh...


on Wed, 19 Dec 2007 09:56:28 captcha'n said:

cool captcha


on Wed, 19 Dec 2007 10:38:29 nigs said:
                                                                      
                _____    ____                                         
             .#########.#######..                                     
          .#######################.                                   
        .############################.                                
       .################################.                             
      .#########,##,#####################.                            
     .#########-#,'########## ############.                           
    .######'#-##' # ##'### #. `####:#######.                          
    #####:'# #'###,##' # #  #. `###:':######                          
   .####,'###,##  ###  # # #`#. -####`######.                         
  .#### .' ,'#  ##' #   # # #`#`.`#####::####   I'VE GOT THIS TERRIBLE
  ####'    #  '##'  #   #_'# #.## `#######;##   CASE OF THE RON PAULSY
  #:##'      '       #   # ``..__# `#######:#                         
  #:##'  .#######s.   #.  .s######.\\`#####:##                         
  #:##   ."______""'    '""_____"". `.#####:#                         
 .#:##   ><'(##)'> )    ( <'(##)`><   `####;#                         
 ##:##  , , -==-' '.    .` `-==- . \\  ######'                         
 #|-'| / /      ,  :    : ,       \\ ` :####:'                         
 :#  |: '  '   /  .     .  .  `    `  |`####                          
 #|  :|   /   '  '       `  \\   . ,   :  #:#                          
 #L. | | ,  /   `.-._ _.-.'   .  \\ \\  : ) J##                         
###\\ `  /  '                   \\  : : |  /##                          
 ## #|.:: '       _    _        ` | | |'####                          
 #####|\\  |  (.-'.__`-'__.`-.)   :| ' ######                          
 ######\\:      `-...___..-' '     :: /######                          
 #######\\``.                   ,'|  /#######                          
.# ######\\  \\       ___       / /' /#######                           
# #'#####|\\  \\    -     -    /  ,'|### #. #.                          
`#  #####| '-.`             ' ,-'  |### #  #                          
    #' `#|    '._         ,.-'     #`#`#.                             
         |       .'------' :       |#   #                             
         |       :         :       |                                  
         |       :         :       |                                  
                 :         :          dp                              

on Wed, 19 Dec 2007 12:42:36 quasi said:

Folks who have no prior knowledge and get nicely introduced to Common Lisp have minimal trouble starting to use it. So the CL coder problem is over rated. If we spend some time educating we get the results. We found a fresher takes about 3 months to start enjoying and doing real work.

Some of us program on GNU/Linux laptops/desktops some on MacBookPro's some on PPC iMac's. We deploy on 64bit GNU/Linux. We have had no issues regarding this setup.


on Wed, 19 Dec 2007 15:29:46 Brian Adkins said:

s/payed/paid/

Also, you might consider using https for the login.

I hope you succeed and become a success story for Lisp.


on Wed, 19 Dec 2007 18:43:22 tieze said:

Thanks Brian, fixed.


on Wed, 19 Dec 2007 20:23:12 NIL said:

NIL


on Wed, 19 Dec 2007 21:02:06 NIL said:

NIL


on Thu, 20 Dec 2007 00:52:25 Victor K said:

.

(defun eratosthenes (n)
  "Return list of primes <= n"
  (let* ((half (ceiling (/ n 2)))
         (numbers (make-array half :initial-element 0)))
    (do ((i 1 (1  i)))
        ((>= i (ceiling (/ (sqrt n) 2))))
      (when (zerop (aref numbers i))
        (do ((j (  i (1  (* 2 i))) (  j (1  (* 2 i)))))
            ((>= j half))
          (setf (aref numbers j) 1))))
    (let ((result (list 2)))
      (do ((i 1 (1  i)))
          ((>= i half) (nreverse result))
        (when (zerop (aref numbers i))
          (push (1  (* 2 i)) result))))))

(defun prime-split (n k &optional (primes nil))
  "Split n into k primes"
  (let ((primes (or primes (nreverse (eratosthenes n)))))
    (cond ((and (= k 1) (member n primes)) (list n))
          ((or (<= k 0) (<= n 1)) nil)
          (t (dolist (p primes nil)
               (let ((ps (prime-split (- n p) (1- k) primes)))
                 (if ps
                     (return (cons p ps)))))))))

on Thu, 20 Dec 2007 01:01:32 Victor K said:

It's strange, but all the '1' in previous comment were replaced with just '1'. Is there any way to post lisp snippets here nicely? :P


on Thu, 20 Dec 2007 06:10:55 tieze said:

no. Clear and firm. And it's not terribly high up on the agenda. It is a bit odd though.


on Sat, 22 Dec 2007 08:52:48 aycan said:

Hey ties,

Take a look at our web server please:

http://www.core.gen.tr/projects/core-server/

Best Regards


on Sun, 23 Dec 2007 11:34:51 Marcin said:

How long did it take to create stix.to?


Want to comment? Type comment. Type something else to randomy chat a bit. I won't bite.
>