Tsar

If you've been working with Linux system for enough time, you will know sar. Since sar supports collecting many system statistic and its syntax is a bit complicated, Yahoo people created a tool in Perl called Ysar, it's similar to sar with simpler syntax, Yahoo has not published that tool yet. People at Taobao created another tool in C, named it with tsar and open source it at: https://gitorious.org/trafficserver/tsar/commits/master


Working with Apache Traffic server to build internal CDN for our company, I also join #trafficserver on FreeNode, that's when I heard about Tsar and started using it.  The opensource version of Tsar has 2 features, once is to get its syntax simpler and easier to use than sar, another is collecting various statistic of Apache Traffic server, integrate with nagios. For some reasons, this public version does not work in some cases: 

• It only support traffic server installed as standard Red Hat package, so if you installed traffic server on another Linux distro,  BSD or Sun Solaris, it will not work. 
• Module traffic does not work.
• it  only supports standard network ethernet (eth*), bonding and p2p device won't work. 

I modified tsar to fix above problems. the modified version will work in any kind of Apache traffic server on any system. It support bonding and p2p and standard ethernet device....

My pull requests was not accepted by Taobao developer, so I put it on github  (Installation and Usage in README file). If you're using traffic server with nagios monitor system, this tool will fit your need. 

Load balancing using HTTP redirect reponse

This May I wrote a simple HTTP load balancer using HTTP 302 redirect response based on Facebook Tornado, it's called rbalancer  
After running it for 2 months, I'm confidence that it's stable and quite good for some purpose.
This's a short introduction of rbalancer:

rbalancer is a simple HTTP load balancer using HTTP 302 redirect response with round-robin and weighted random support built ontornado. rbalancer can perform health check on server in balancer list, when a server went down, rbalancer will automatic re-balance request to alive servers. When dead one goes up, it will be added to cluster and serving request likes normal

If you're curious  about this thing, come check it on github

PyLogagent

I wrote a small python logging agent to work around with application log files which cannot be logged to central log server (Syslog-Ng)

Source code is available on Github
----
Some log files generated by application cannot be logged to syslog/syslog-ng

pylogagent will watch list of log files in "configuration" file, then push new lines to /dev/log socket with prefix.

USAGE:
1. Set up syslog-ng server to listen for log

----snip-----
source net {
udp(ip(ip_of_log_server) port(514));
};
destination nginx_error {
file("/data/log/$HOST_FROM/$YEAR/$MONTH/$DAY/nginx_error");
};
filter nginx_error {
match("^nginx_error");
};
log {source(net);filter(nginx_error);destination(nginx_error;};
-----snip-----

These line above will make syslog-ng server listen for log on udp port 514, whenever a log stream deliver to its port, syslog-ng will parse, look for regex ^nginx_error, if it is found, log to file nginx_error.


1. Configure which log files pylogagent will watch

#configuration
/data/log/nginx_error.log

2. Start logagent
# python agent 

3. Configure syslog-ng client

------snip--------
source s_local {
internal();
unix-stream("/dev/log");
};
destination d_loghost {
udp("ip_of_log_server" port(514));
};
filter nginx_error            {
match("^nginx_error");
};
log {
source(s_local);
filter(nginx_error);
destination(d_loghost);
};

------snip--------

Restart syslog-ng and you are done.
Tail log file on central log to see how it works ;)



--
More will be added later

Interesting stuff

What I've got to read today:

• Python Patterns - An Optimization Anecdote
• URLs in C (Funny)
• Solving algorithm problems vs. editing configuration

Suddenly realize I have too much time to read these days ;) . 

Python for Informatics: Exploring Information

If you're looking for the way learning Python on your own, and still can't find a good document, book or lecturer, this online course should be a right choice.

The goal of this site is to provide a set of materials to allow you to learn Python on your own. This page serves as an outline of the materials to support the textbook "Python for Informatics: Exploring Information". The materials are focused on the "core-programming skills" contained in Chapters 1-10. The slides, video lectures, and worked exercises are prepared specifically for online use. The "Lecture Audio" is recorded from the SI502 - Networked Computing course at the University of Michigan 

http://prezage.com/player/?p=40

Gentoo on Thinkpad T410: authen using fingerprint (gdm)

Step 1:
Download 2 ebuild files named: fingerprint-gui and upekbsapi-bin from this website http://gpo.zugaina.org.
Step 2: Create digest manifest file for them

cd /usr/portage/sys-auth
mkdir {fingerprint-gui, upekbsapi-bin} 

Copy fingerprint-gui-1.00.ebuild and fingerprint-gui-1.00.ebuild to these directories.

ebuild fingerprint-gui-1.00.ebuild digest 
ebuild  upekbsapi-bin-4.0.0218.ebuild digest 

Step 3: unmark  using package.keywords and install

emerge sys-auth/fingerprint-gui 

it shoud install both fingerprint-gui and upekbsapi-bin

Step 4: unmark libfprint and install it
Step 5: load fingerprint-gui and follow the installation steps to scan and verify your finger.
Step 6: Adding fingerprint-gui into pam and gdm

cat /etc/pam.d/system-auth
auth        required    pam_env.so
auth        sufficient  pam_fingerprint-gui.so try_first_identified
auth        required    pam_unix.so try_first_pass likeauth nullok
auth        optional    pam_permit.so
account        required    pam_unix.so
account        optional    pam_permit.so


cat /etc/pam.d/gdm
#%PAM-1.0
auth optional pam_fingerprint-gui.so -d
auth       optional        pam_env.so
auth       include        system-login
auth       required        pam_nologin.so

account    include        system-login

Logout and login to test new authentication method ;-)

Life definition

What is a wing? How does it make something fly? It’s the same exact type basic materials (quarks, protons, electrons, etc.) simply in another configuration. So do wings really make things fly?
How about a memory? A replaying of an event in time within the mind of a human. This is an organization as well — in this case an organization of chemicals (themselves organization of molecules of atoms of quarks) that allow us to store information in a format that is similar to that of human experience.
What is human experience? It’s perceived using the brain (another organization) and a number of sensory organs that are themselves specialized organizations of building blocks.
Fundamentally, we have things crashing into each other at the subatomic level, and everything else (the things we give names to and understand) are nothing more than concepts associated with the organization of said collisions.
That’s what life and death is — the collection of some of these bits into a formation that matches a particular type of concept until it doesn’t match that pattern anymore.
If we build a sand castle from sand, and then it gets destroyed by the rising tide, where did that castle go? Where did it come from? Did it actually exist at all? What is, “castle”?
What is life? What is love?
It is like “human”. They are all like the castle — temporary organizations of building blocks in a pattern recognized. And once they are not in that pattern anymore they “die”.
But nothing dies because nothing ever existed. There’s just a bunch of stuff banging into itself.

Original post