Node.js – building massively parallel applications with javascript and epoll

Node.js [http://nodejs.org/] is a framework for rapidly building massively parallel network applications using non-blocking io. It’s built on the performant v8 javascript engine [http://code.google.com/p/v8/] which compiles javascript to machine code at runtime. Node is similar in design to systems like Ruby’s EventMachine [http://rubyforge.org/projects/eventmachine/] or python’s twisted [http://twistedmatrix.com/trac/]. It’s suitable for building chat-like protocols, comet servers, mmo game servers, etc. This session will cover some of the basics of how to build and install node.js as well as analyze some demo programs.

If time permits, I might talk a little bit about python’s twisted and compare some programs written using twisted/node.js

About Shane Hansen:

I’m a Software Engineer working on Backcountry.com’s replatforming teams. Prior to that I worked on projects ranging from compliance tracking software for the goverment to building  mathematical models for reflectometry to immune system modeling. I’m interested in open source software, web standards, high performance computing and application architecture

Meeting Location:

BetaLoft (http://betaloftslc.com) is located at 357 W 200 S. Upstairs in Suite 201. Salt Lake City, UT, 84101

Betaloft is a coworking space in the heart of downtown Salt Lake City founded by Drew Tyler.

We are a community of freelancers, independents and work-from-home people, who have an interest in getting out of our dull work day and participating in a community that’s designed to support and encourage people who make a living doing what they love.

We provide communication tools, events (social and educational), as well as a physical space to work and collaborate.

See you all there!

Clint

Today, I found myself needing to enable the network (using dhcp) from the anaconda command line (the part that says boot: when you load a CentOS/RHEL/Fedora install disk).

I needed this because I was providing a bit of local media to our external employees for them to use to install, but I had 3 yum repositories I wanted them to be able to use for a few additional rpms I provided.

So, I popped into one of my common support channels and asked the question:

16:11 < herlo> looking for an option that I can put on the anaconda boot: prompt that will force a
               dhcp request even though the install is from a local disk.  I have repos that are not
               being accessed because the network is not being enabled.
16:11 < herlo> things I have tried
16:11 < herlo> ip=dhcp
16:12 < herlo> boot: linux noipv6 ks=hd:sdb1:media/kickstart.cfg ip=dhcp
16:20 < herlo> okay, so for those who might care
16:20 < herlo> if you do
16:20 < herlo> boot: linux asknetwork ip=dhcp noipv6 ks=hd:sdb1:media/kickstart.cfg
16:20 < herlo> anaconda will force a network dhcp request :)
16:20  * herlo is happy again

As you may have noticed above, I resolved this issue without any assistance from the channel, and it only took me 10 minutes to do so.  I found my answer by trial and error from a great page on the fedora wiki: https://fedoraproject.org/wiki/Anaconda/Options

Probably the only thing I wish was documented on that page was when each of the options started being supported in anaconda.  Otherwise, thank you to the folks that wrote and maintain that page.

Cheers,

Herlo

Vim 101 and Beyond

Vim is the editor of choice for many developers and power users. It’s a “modal” text editor based on the vi editor written by Bill Joy in the 1970s for a version of UNIX. It inherits the key bindings of vi, but also adds a great deal of functionality and extensibility that are missing from the original vi. This session will cover the beginnings (101) and the first set of intermediate (201) modes and commands for vim.

If you would like to join us, just head on down via train, car (parking is free on the street for two hours just east and west of BetaLoftSLC), walk or bike to:

WHEN

Wednesday, February 10, 2010
11:30am-1:00pm

WHERE

Come learn some tips and tricks about Vim.  We hope to see you all there.

Cheers,

Clint

Friday, January 29 — Day 1

Gathered at Red Hat’s main office, we brainstormed in a manual ‘tag cloud’ kind of way.  Mel had us all take sticky notes, write upon them based upon a few words on the white board and then, stick them to said white board appropriately.  This got our minds going about what a FUDCon or FAD should be, why it was important and the things that could really be improved.  I felt very happy about the amount of ideas that were shared on these sticky notes.  it was quite cathartic to get out the things that always had bugged me or I thought needed improvement in our Events.  I have a few pictures of us doing this process, enjoy them.

After spending about 1.5 hours doing this and discussing it, we broke into separate groups, the FUDCon 2.0 folks (upstairs) and FUDCon Live folks, aka me, Yaakov, and the freeseer folks online (downstairs).  My main target was to get the freeseer application working with completely free software and build the AV Kit from components I had, plus the ones that Mel had purchased for this project.

After getting downstairs with Dennis Gilmore (he was my helper for the first hour), we quickly discovered that one component, the Epiphan vga2usb device, was not working.  After a bit of digging, we also discovered that it had a non-free driver and that it would likely not be easy to find a free driver alternative.  We did, however, attempt to build the binary they provided, but kept getting errors.  More on this later on (or in another post), so stay tuned.

I spent the next few hours trying to get everything else up and running, doing research to find a different alternative for video output from a VGA source to USB input.  heffer joined us on IRC and gave me some good links as to where I might look for a Scan Converter and a easyCAP device.  While a little lower quality, the Linux drivers for it are completely open and free, so I set out with a plan to find one in Raleigh.

At 4pm, Max and I headed out on the town, hunting down several items, including firewire PCMCIA adapters (for our miniDV camera) and the Scan Converter components.  We needed to get a screw driver and some other firewire adapter stuff too, we headed to CompUSA. Though normally I wouldn’t go there, but this CompUSA had actually been converted from a Tiger Direct, so I thought we had a chance.  After about 1.5 hours of failure, we ended up with two firewire cards and some audio cables, we headed off to see Avatar in 3D.

Saturday, January 30 — Day 2

After leaving Avatar, we discovered a nice big blanket of snow had come down in Raleigh.  Just 2-3 inches, and in Utah, we’d think nothing of this, but here it’s quite a bit different.  First off, North Carolina doesn’t seems to have the infrastructure, no plows or ice melt, to really deal with something like this kind of storm.  There were news reports of it on every station, the Governor called for a state of emergency, and I just thought it was odd.  Because of this, it was determined that we would not leave the hotel for Day 2 of the FAD.  Instead, we reserved a room in the hotel and worked from there.  Luckily, the hotels infrastructure, plus the Days Inn next door provided us with our networking needs, while Max stayed at his apartment and called in using Fedora Talk.

My work was to spend as much time with the FreeSeeR folks and do tons and tons of testing of their code, plus provide feedback and gstreamer pipelines to get us closer and closer to our eventual goal.  Thanh had spent a lot of time while we were at Avatar to turn FreeSeeR into an API.  He also altered the code to put the gui into a more sensible tool, with both Qt and Gtk implementations.

About half way through the day, I discovered that I had accidentally left my power adapter for my audio mixer in Max’s car (he was 15 minutes way with no snow and at least 25 with), essentially eliminating my high quality audio testing.  Luckily Chris Tyler had a headset with a microphone and Dennis Gilmore had a webcam we could use because the firewire cards were a bit flaky and kept crashing my kernel.

By the end of the night, with some tweaking by Dennis and I, we had FreeSeeR working with DV input, USB video input, 1/4″ audio input and were able to output to an ogg file with reasonable quality and consistency.  A lot of testing later, and we were able to determine that we still needed to tweak some of the code to provide for a better way to adjust audio and video settings prior to recording.  All in all, the FreeSeeR software is coming along very nicely.  Andrew Ross and Thanh Ha have been doing an amazing job and I really appreciate their help working on getting this working.  The new version of FUDCon Live thanks you as well, because without this, we won’t be able to provide our users with a good quality remote experience.

Sunday, January 31 — Day 3

The sun is shining, but for some reason, the roads are still not that clear.  Several cars are still having difficulty climbing the incline out of the Best Western to the main road, which is now melting, but still very snow covered.  Today, we discover that we’ve met one major part of our goals, the Fedora Pony has been created!!  We must thank Robyn Bergeron for creating, Frank, the Fedora Pwnie.  Now mind you, Francis is really her name, but she’s such a tomboy that, well, you just can’t call her that, she doesn’t enjoy it too much.  So we call her Frank.

In addition to our major goal above of a Pony, the FUDCon Live team has done some amazing work.  Yaakov has been working on the FUDCon Live document with Mel, while I was working with the FreeSeeR guys to get their git repo moved over to fedorahosted.org, which is awesome!  I’ve been given commit and sponsor rights to the repo, so we’ll start getting more developers involved right away.  Have a look at the screenshots of the GUI if you’d like to see what FreeSeeR can do.

Jon Stanley and I discussed the possibility of moving fedorahosted.org over to gitolite, and discovered that Jesse Keating has been experimenting with it himself, so this might be something we can do in the near future.  While we currently appear to use gitosis, gitolite gives us the ability to set ACLs on a particular branch, which then can help keep the master branch cleaner.  To help illustrate this, there’s a very great article on nvie.com which explains a git branching system which can really make development and commits very clean and easy to track.  Gitolite can help with this, so I’m going to be experimenting with it this coming week.

I spent the rest of the day writing up the AV Kit wiki page along with Mel.  I stubbed it out, and she added a big section regarding the modules in the AV Kit.  I then rewrote much of that to cover the two styles of AV Kit we’ll be building over the next month or two.  In fact, I plan to have one complete in time for the Marketing FAD in March, where they can use and test it out.  I really hope to get some good feedback on it and improve FreeSeeR some more using these upcoming events as testing grounds.

Currently, I’m on a plane which I didn’t think would take off tonight, headed home for Salt Lake City.  I’m excited to see my sweetie and get some much needed sleep.  As much as I enjoy hanging out with my Fedora friends and working on projects like this, it really wears me out.  I’m ecstatic at the amount of work we accomplished though, and am very appreciative to Paul, Jon, Chris, Denis, Dave, Mel and Max, plus all the folks online for their hard work this weekend.

FUDCon 2.0 is alive and kicking, FUDCon Live will make it just that much better.  Watch for upcoming posts in the near future regarding FreeSeeR and the Fedora AV Kit and how everything is going to work.

Cheers,

Herlo

It’s clear that Mel has done some incredible work, getting a good number of people to Raleigh this weekend.  I’m looking forward to giving my input and listenting to others as we come up with better ways to handle such things as the EventBox, recording and streaming different presentations, encouraging users to join Fedora and the overall professionalism we portray at each of our events.  There are so many things to get done this weekend, I’m looking forward to seeing what comes out the other side.

As for what I will be doing most of the time.  While I plan to help with the initial brainstorming on FUDCon, external events, etc., I really plan to spend most of my time focusing on improving the way that we record and stream our events.  These events can be as simple as a Local User Group (LUG) meeting to something bigger, like the Southern California Linux Expo (SCaLE), the Ohio Linux Fest or Pycon, and of course, somewhere in there, is the Fedora FAD and FUDCon.

Now mind you, I can see hundreds of other uses for this concept of recording and streaming.  Including live video podcasts, recording for redistribution, ad-hoc collaboration sessions, troubleshooting a problem with code and many, many more use cases.  However, I think I’m going to try to limit my use cases to those specifically around our events.  Things where we can easily set up our recording equipment and share it with those interested, whether in real-time, or making it available later on to the general public.

The software we plan on using to get the recording and streaming off the ground, is called freeseer.  It’s been put together by some folks who help organize and run FOSSLC.  Andrew and Thanh have been hacking hard lately, altering their code to move from the patent encumbered ffmpeg, to the more open and free gstreamer library.  They’ve also been busy moving from a gui with a command line behind it, to using the python gstreamer bindings.  Much cleaner, much easier to manage and alter in real time.  I’m very excited to see how we can improve freeseer over this weekend.  I actually think we’ll be able to do quite a bit with freeseer to improve external participation in Fedora.

Well, the pilot just announced that we’ll be landing in Raleigh in about 20-30 minutes, and that I need to pack up my electronics.  I’m looking forward to seeing all of my Fedora friends and making an amazing events solution.  I’ll try to keep you updated, with pictures and text, over the next few days.

Cheers,

Herlo

As usual, come from 11:30am-1pm at BetaLoftSLC (357 W 200 S #201)  - http://betaloftslc.com

This month, I thought I would share a basic git introduction.  In my opinion, the best Source Control Management (SCM) system out right now.  I’ve been using it for nigh on a year and I have converted a few systems over to it from svn.  This is more of a work session than a presentation, so come on down and learn how to use git.  It is recommended that you bring your laptops with you as that will benefit more if you do.

Some suggested topics I can cover:

• Local and remote git repositories (different workflows and other fun stuff)
• Using git-svn to ‘convert’ from svn
• Comparing Distributed Revision Control with Traditional (non-distributed) Revision Control
• Git hooks, why they are valuable and when to use them
• Other basic git discussions

See you all there.

Cheers,

Herlo

I am very sad to report, there will not be any pressed F12 media at FUDCon this time.

However, I expect the financial details to be sorted out sometime between now and Monday.  If this happens, it’s quite possible we can deliver media starting next week.  Still pretty fast, considering.

I guess every FUDCon can’t be perfect, but we’re still going to make this a great event!

If you are in the area, or wish to participate from afar, we’ve got ways for you to get involved.  FUDCon is always free and I’ve got two remote microphones and two mixers that I plan on using to record and stream the audio from two rooms.  I’m also working on a way to stream/record the desktop of the presenter at the same time so we can have audio and video from the desktops in real time (we’ll see if this part happens).  I might even suggest a hackfest on Monday to make it more solid if I can’t get it working tonight/tomorrow.

I’m excited to see all my friends at FUDCon.

Cheers,

Clint

Everything Just Works(TM).

I highly recommend it to anyone who wants to have a system working with great sound, awesome graphics and very usable GNOME and KDE desktops.  I really like the new NetworkManager features and the ability to tether any device, vpnc and other easy-to-configure options.  KVM support is much improved with the new libguestfs improvements.

Fedora 12 media should be available on December 3, if you are a Fedora Ambassador in North America, start requesting media* for your events (and ambassador kits).

You can get your copy today!!

Cheers,

Herlo

*NOTE: This request page is not for the freemedia project. You must be a Fedora Contributor to request media.  Join the Fedora Project today and help us continue making Fedora the best operating system!!

RSSCloud is an open protocol which creates a loosely-coupled Twitter-like network of people and 140 character status messages.  In plainer language, it enables real-time status updates on anything that can be pushed via RSS.  Why should you care?  Come to the SLLUG daytime SIG meeting to find out.

More information can be found at http://rsscloud.org/

So why not come down to BetaLoft next Wednesday, November 11, from 11:30am-1pm and learn about RSSCloud.

Cheers,

Herlo

To do this, I’ve been using a centralized git repository using gitosis and gitweb.  Because it’s behind the Backcountry firewalls, there is no easy way to share our repository data.  However, there are some great git repos at github, which I highly recommend.

Anyway, we do the hub-and-spoke model with our git repositories.  Meaning that we have a central repository where much of the data is stored so everyone can get access.  Of course, though, most people clone these repositories and work on the code where they choose.  Because we have several people pushing puppet data to our repository, which has been on subversion until 2 weeks ago, I am keen on making sure the puppet code we push to the central repo is at least syntactically valid.

Git Hooks: The skinny

Essentially, the key here is to understand is that git, like many other popular revision control systems is distributed.  This means that many people keep repos locally and push to the central repository after many many commits (or just one or two).  Git keeps the history in a clean way and all of the commits are pushed and tracked.  Essentially, we need some way to validate the data coming in from an outside source.  Enter git hooks.

There are several git hooks, but three that are quite critical to this scenario; pre-commit, pre-receive and post-receive.

The first, pre-commit is specific to the repository where the commit is taking place, such that a validation of the puppet syntax here is useful to help the person writing the puppet rules.

Next is post-receive which is generally useful for sending out emails after a successful push.  It would list the files and the diff of each in an email to those who wish to know about the commit.

And finally, the least documented of the bunch; the one with the least amount of examples, pre-receive.  The pre-receive hook has some really awesome power, but without knowing the git commands backward and forward, it can be very frustrating to get working.

The value of the pre-receive hook

Whenever a push is made to the centralized repository, you don’t just want to let any willy nilly commit crawl in there.  You can’t trust that the author of the commits did his part to validate the code.  So you need to at least do some basic checks.  That is what the pre-receive hook does.  In the following example, I am parsing files that end in ‘.pp’ with puppet syntax validation as you would do from the command line.

#!/bin/bash

while read old_sha1 new_sha1 refname anothervalue ; do
    list=$(git show --pretty="format:" --name-only $new_sha1 | grep -e ".pp$")
    for tmpfile in ${list}; do
        git show ${new_sha1}:${tmpfile} | puppet --color=false --confdir=/tmp --vardir=/tmp --parseonly --ignoreimport
        if [ "$?" -ne "0" ]; then exit 1; fi
    done
done

exit $?

The above script, placed in a bare repository at hooks/pre-receive with the execute (+x) bit turned on, will get a list of files that end in ‘.pp’ and run the puppet validator for the newest commit in the push.  Essentially, I can validate someone else’s puppet code in just a few seconds and return a failure if it doesn’t succeed.  With git hooks, a failure means no commit/push/etc will actually happen.

I sure hope this helps someone else to understand pre-receive hooks as well as git hooks in general.  If you have any helpful suggestions or comments, those are most welcome as well.

Cheers,

Herlo