Edward A. Webb (.com) » MIsc.Tips

Verify file Checksum in Windows with context menu

Eddie — Tue, 07 Jun 2011 00:54:47 +0000

Checking the integrity of a file on a unix node is simple thanks to a confusingly named “checksum” utility. It, oddly enough gives you the checksum of a file.

If your stuck on (or just love) windows, you can give yourself the ability to check any files checksum with the click of a mouse.

Right-Click any file to validate the checksum integrity

Download Microsoft’s FCIV.exe

You can download their unsupported command line utility fciv from the following link, http://support.microsoft.com/kb/841290.

If you have another tool, you may substitute and move on.

Create a simple Batch file

..to run the checksum utility of your choice, and then pause

@echo off
:: edward a webb
:: June 6 2011
:: runs micorsofts fciv command line tool against file and wiats, fort use as context menu
:: http://edwardawebb.com/?p=955
cls
fciv.exe %1
pause

Drop both onto your PATH

You can either add them to you system variables , or drop them in a folder like :\windows\system 32. I’m lazy and did the latter.

Add the batch file as an context menu option

Just save the file below, right click and merge into your registry. It makes any file’s right click menu include the value of the first key as the text you’ll see, and the value of the second (“command”) key, the actual command to be run. Substitute “%1″ for the file (yes use double-quotes escaped in case the file your checking is on a path with spaces).

Windows Registry Editor Version 5.00
 
[HKEY_CLASSES_ROOT\*\shell\Checksum]
@="Verify CHecksum"
 
[HKEY_CLASSES_ROOT\*\shell\Checksum\Command]
@="verify-checksum.bat \"%1\""

That’s it, the changes are effective immediately, so try it out.

Getting IP addresses for remote nodes

Eddie — Thu, 29 Jul 2010 15:14:07 +0000

I recently had the need to get the ip address for a remote node. Wait, scratch that.. I had to get the IP addresses for about 60 remote nodes.

You’re right… I probably could have typed

nslookup targetNode

60 times. But I am a programmer! SO I decided to pass my choir off to a shell script.

If you have a similar need, read on!

The script is simple, just add all target nodes as a space separated list, and run the file.

#!/bin/bash 
# author: Edward A Webb (http://edwardawebb.com
# license: Just use it! its like 10 lines of shell script...
 
# add your actual nodes here.....
nodes=( gonzo fenway einstein yogurt toxic babmbi rambo donna INXS etc... )
 
ips=( ) # placeholder for results
 
printf "\t Determing IP addresses for provided nodes \n ignore these lines...\n"
for (( i = 0 ; i < ${#nodes[@]} ; i++ ))
do
	str=$(nslookup ${nodes[$i]} | tail --lines=2 | head --lines=1)
	ips[$i]=$(echo $str | cut -f 2 -d " ")
 
done
 
printf "\t COMPLETE\n Here are the results you care about..\n"
 
for (( i = 0 ; i < ${#ips[@]} ; i++ ))
do
	echo address for ${nodes[$i]} is ${ips[$i]} | tee -a ip_results.txt
done

Passing Parameters with h:commandLink in JSF

Eddie — Tue, 13 Jul 2010 19:47:57 +0000

I have been growing fond of JSF as of late, but was stumped by the simplest task.

How do I associate some value with a link? If I was using PHP I would just append a query parameter and grab the value. But with JSF it’s all done inside our xhtml files, so how do we set a value based on the link clicked?

Just use

inside the commandAction.

This stores any expression into the specified bean’s property.

See below for a full faces example. Each Player link will set the backing Bean’s player object before calling the action.


 
pageTitle
appTitle
content
 
 

	Team Management
 
 
	Affinity Management Plus (Basketball)
 
	
	
		 #{team.teamName}  Player Summary
		
		
                  
                       -  
 
                  
          	
		  	
 
		 #{team.teamName} Game Summary
 
		
 
                 Played #{game.date} against #{game.team2.teamName}
 
				 
 
					
 
                 	Stats for 
                 	#{entry.key.name}
                 	  
                 		
					
					    
					        1
					        
					        2
					        
					        3
					        
					        fouls	
					        
					        steals
					        
					        rebounds
					        
					        minutes
					        
					    
					    
					        #{entry.value.onePointers}
					        
					        #{entry.value.twoPointers}
					        
					        #{entry.value.threePointers}
					        
					        #{entry.value.fouls}
					        
					        #{entry.value.steals}
					        
					        #{entry.value.rebounds}
					        
					        #{entry.value.minutesPlayed}

1	2	3	fouls	steals	rebounds	minutes
#{entry.value.onePointers}	#{entry.value.twoPointers}	#{entry.value.threePointers}	#{entry.value.fouls}	#{entry.value.steals}	#{entry.value.rebounds}	#{entry.value.minutesPlayed}

 
 
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
 
import javax.faces.event.ActionEvent;
import javax.faces.model.SelectItem;
 
import com.libertymutual.amp.basketball.DAO.GameDAO;
import com.libertymutual.amp.basketball.DAO.PlayerDAO;
import com.libertymutual.amp.basketball.DAO.TeamDAO;
 
public class TeamProcessor {
	//Business MOdels (used for temporary associations and datastore)
	private Team team= new Team();
	private Player player=new Player();
	private Game game=new Game();
	private Statistic statistic = new Statistic();
 
	private CumulativeStatistic cumaltiveStat= new CumulativeStatistic();
 
	private Team team2= new Team();
	private List<SelectItem> opponents = new LinkedList<SelectItem>();
	private TeamDAO teamDAO = new TeamDAO();
	private PlayerDAO playerDAO = new PlayerDAO();
	private GameDAO gameDAO = new GameDAO();
 
	//authorization
	private List<User> users= new LinkedList<User>();
	private String tmpName="";
	private String tmpPass="";
	private boolean isAuthorized=false;
 
 
	//expose properties to jsf
	public Game getGame() {
		return game;
	}
 
	public void setGame(Game game) {
		this.game = game;
	}
 
	public Statistic getStatistic() {
		return statistic;
	}
 
	public void setStatistic(Statistic statistic) {
		this.statistic = statistic;
	}
 
	public List<User> getUsers() {
		return users;
	}
 
	public void setUsers(List<User> users) {
		this.users = users;
	}
 
	public String getTmpName() {
		return tmpName;
	}
 
	public void setTmpName(String tmpName) {
		this.tmpName = tmpName;
	}
 
	public String getTmpPass() {
		return tmpPass;
	}
 
	public void setTmpPass(String tmpPass) {
		this.tmpPass = tmpPass;
	}
 
	public Player getPlayer() {
		return player;
	}
 
	public void setPlayer(Player player) {
		this.player = player;
	}
 
	public  Team getTeam() {
		return team;
	}
 
	public void setTeam(Team team) {
		this.team = team;
	}
 
	public void setTeam2(Team team2) {
		this.team2 = team2;
	}
 
	public Team getTeam2() {
		return team2;
	}
 
	public List<SelectItem> getOpponents() {
		return opponents;
	}
 
	public void setOpponents(List<SelectItem> opponents) {
		this.opponents = opponents;
	}
 
	public TeamProcessor(){
		// set valid users
		User eddie = new User();
		eddie.setName("eddie");
		eddie.setPassword("1234");
		users.add(eddie);
 
 
		opponents = new LinkedList<SelectItem>();
        for (Team team : teamDAO.list()) {
            opponents.add(new SelectItem(team,team.getTeamName()));
        }
 
 
	}
 
	/*
	 * 
	 * Action Methods
	 */
 
	public String processTeam(){
		if(!isAuthorized)return "invalid-login";
		String result= "success";
		return result ;
	}
 
	public String processPlayer(){
		System.out.println(">>> processPlayer");
		if(!isAuthorized)return "invalid-login";
		String result= "error";
		if( team.addPlayer(player)){
			result="success";
			playerDAO.addPlayer(player);
			System.out.println("Added: "+player.getName());
		}
		player=new Player();
		System.out.println("<<< processPlayer");
 
		return result;
	}
 
	public String processPlayerAndAddAnother(){	
		System.out.println(">>> processPlayerAndAddAnother");
		if(!isAuthorized)return "invalid-login";
		processPlayer() ;
		System.out.println("<<< processPlayerAndAddAnother");
 
		return null;
	}
	public String processPlayerDoner(){	
		return "success";
	}
 
 
	public String loginAction(){
		String result="invalid-login";
		for(User user : users){
			if(tmpName.equals(user.getName())
					&&
			tmpPass.equals(user.getPassword())){
				result = "success";
				isAuthorized=true;
			}
		}
		return result;
	}
 
	public String processGame(){
		System.out.println(">>> processGame");
		String result="error";
		game.setTeam1(this.team);
		game.setTeam2(team2);
		if(team.addGame(game)){
			gameDAO.addGame(game);
			result="success";
			game=new Game();
		}
 
		System.out.println("<<< processGame:"+result);
 
		return result;
	}
 
	public String processGameStats(){
 
		game.addGameStat(player, statistic);
		System.out.println(">>> processGameStats: " + game.getTeam2().getTeamName()+player.getName()+statistic.getFouls());
		player=new Player();
		statistic=new Statistic();
		game=new Game();
		return null;
 
	}
 
	public String finishGameStats(){
		processGameStats();
		return "success";
	}
 
	public String preparePlayerAction(){
		String result = "error";
		System.out.println("playerAction>>>");
 
		System.out.println("Player:"+player.toString());
		System.out.println("team:"+team.toString());
		for( Game game : team.getGames()){
			Statistic stat = game.getGameStatistic(player);
			if (stat != null){
				cumaltiveStat.addStatistic(stat);
			}
			result="success";
		}		
		System.out.println("CS:"+cumaltiveStat.getAverage(cumaltiveStat.getFouls()));
		System.out.println("playerAction<<<");
		return result;
	}
 
	public void preparePlayerListener(ActionEvent ae){
		System.out.println("playerListener>>>");
		for (Iterator iterator = ae.getComponent().getAttributes().entrySet().iterator(); iterator.hasNext();) {
			Object o = iterator.next();
			System.out.println(o.toString());
 
		}
		System.out.println("Player:"+player.toString());
		System.out.println("playerListener<<<");
	}
 
	public CumulativeStatistic getCumaltiveStat() {
		return cumaltiveStat;
	}
 
	public void setCumaltiveStat(CumulativeStatistic cumaltiveStat) {
		this.cumaltiveStat = cumaltiveStat;
	}
 
 
 
 
 
 
 
 
}

Plugins for a Full Featured WordPress Content Manager

Eddie — Tue, 04 May 2010 14:06:05 +0000

More and more clients are asking for dynamic sites that allow them to edit specials and pages at will. This is great because it surely means they will come back for more questions and changes as they use the site. (read : more income).

What they want are CMS, or content management systems. What I give them is usually wordpress loaded with useful plugins. (If custom business logic is needed i turn to CakePHP).

ANd why not? The WordPress engine provides post and page management, RSS feeds and discussion features, and customizable “widgets.” But a few key features are needed to make the leap to a decent CMS.

Read This
(Update: FInally a CMS that clients can use! – found concrete5 CMS very recently, and first impression is wow!- trust me its worth a demo )

Besides, no matter how many times I explain creating a new page in drupal, or how to add a side box in joomla. There is just too many steps and switches for the average small business owner. (is that a huge stereotype? yes, I apologize..) But honestly, most non-technical users don’t think in terms of elements like footer, and side bars, and such. they think of pages! (and conrete5 does that in a spectaciular manner)

..but if you don’t want to give that a shot first, read on.

The plugins handle 4 critical areas for any decent site:

Page and Menu Management
SEO and Meta data
Sitemaps
Web 2.0 / Bookmarking / Sharing

SO what plugins come standard on a Edward A Webb hosted WordPress install? Read more to bathe in the glory of these awesome plugins.

Page Management

WordPress has great page management, but the connection to Menus is lacking. Clients need to hide and reorder pages constantly, and explaining the “weighted order” field is not worth the effort.

PageMash

Customise the order your pages are listed in and manage the parent structure with this simple ajax drag-and-drop administrative interface with an option to toggle the page to be hidden from output. Great tool to quickly re-arrange your page menus.

SEO Enhancement

Yes I can use buzz words. But SEO is legitimately important when administering a site for any business. So to maximize client click-throughs and search ranking I use a few plugins to add clean urls and descriptive tags to everything!

All in One SEO Pack
- Support for CMS-style WordPress installations
- Automatically optimizes your titles for search engines
- Generates META tags automatically
- Avoids the typical duplicate content found on WordPress blogs
- For beginners, you don’t even have to look at the options, it works out-of-the-box. Just install.
SEO Friendly Images

SEO Friendly Images is a WordPress optimization plugin which automatically updates all images with proper ALT and TITLE attributes. If your images do not have ALT and TITLE already set, SEO Friendly Images will add them according the options you set. Additionally this makes the post W3C/xHTML valid as well.
SEO Slugs

The SEO Slugs WordPress plugin removes common words like ‘a’, ‘the’, ‘in’ from post slugs to improve search engine optimization.

Sitemaps

Technically a perk for SEO, but the plugin I use exposes a nice user readable sitemap as well.

Google XML Sitemaps

This plugin will generate a special XML sitemap which will help search engines like Google, Bing, Yahoo and Ask.com to better index your blog. With such a sitemap, it’s much easier for the crawlers to see the complete structure of your site and retrieve it more efficiently.

Social Media / Bookmarking

If you have a group of dedicated followers – they can be your biggest tool for grabbing more readers. But you have to make their job easy! That’s why I recommend a simple sharing plugin for any wordpress site.

SexyBookmarks

Though the name may be a little “edgy” for some, SexyBookmarks has proven time and time again to be an extremely useful and successful tool in getting your readers to actually submit your articles to numerous social bookmarking sites.
WP to Twitter

The WP-to-Twitter plugin posts a Twitter status update from your WordPress blog using either the Cli.gs or Bit.ly URL shortening services to provide a link back to your post from Twitter.
You can also use WordPress’ built in ?p=529 for short urls.

That’s it! Just install those 7 plugins and you’ll be running the next Tumblr.. I swear. Ok well that might be an overstatement, or understatement.. but it’s a good goal to shoot for.

Feedback

How about you? Am I missing any great ones, have you found ones that work better than my listed choices? Please share!

Script It?

You didn’t honestly expect me to expect you to download 7 or ore zip files and upload them to a wp install did you? Hell no, I won’t even make you curl them individually from your server, just add the scripts you want to the plugins array and run the bash script below against your wp install!

$ sh wp_plugins.sh path/to/wp/root

#! /bin/bash
 
# upload plugins to wordpress site
# edward a webb - May 7, 2010
#  http://edwardawebb.com/web-development/7-plugins-full-featured-wordpress-content-manager
 
plugins=( all-in-one-seo-pack google-sitemap-generator.3.2.3 pagemash )
 
 
if [ $# -eq 1 ]
then
	wproot=$1
else
	echo "Usage: $0 path/to/wp/root"
	exit 1
fi
 
if [ -d $wproot ]
then
 
	for (( i = 0 ; i < ${#plugins[@]} ; i++ ))
	do
		curl http://downloads.wordpress.org/plugin/${plugins[$i]}.zip -o ${plugins[$i]}.zip
		unzip ${plugins[$i]}.zip -d $wproot/wp-content/plugins/
		rm -f ${plugins[$i]}.zip
 
 
	done  
else
	printf "\t Could not locate $wproot\n"
	exit 2
 
fi

Backup all sub-directories with a Bash array loop

Eddie — Thu, 15 Apr 2010 22:02:05 +0000

I manage lots of domains, and offer my clients free backup and recovery service.
Nice selling perk, but I best be damn sure I am backing things up regularly. Since there is no way my space-cadet brain would remember that, I rely on my ‘nix friends: bash, cron and tar to neatly package every sub-directory of my webroot into their own little tarballs.

The bash script included after the break reads all directories into an array that we can loop through and manipulate as needed.

Background

My webroot is pretty straightforward. Each subdirectory is the full URL of a site. Example:

webroot
|-- baungenjar.com
|-- bythebootstrap.com
|-- cafe--espresso.com
|-- crm.webbmaster.org
|-- eaw-technologies.com
|-- edwardawebb.com
|-- entrepreneurmeetcapitalist.com
|-- kicksareforribs.com
|-- ollitech.com
|-- outlaw-photography.com
|-- piwik.baungenjar.com
|-- rendinaro.com
|-- riversidegrillnh.com
|-- sidi.webbmaster.org
|-- status.webbmaster.org
|-- steadfastsites.com
|-- stonybrookpottery.com
|-- survey.webbmaster.org
|-- top.webbmaster.org
|-- trippymedia.com
|-- webbmaster.org
`-- wishlist.webbmaster.org

So I want to take each directory and create a gzipped tarball using tar -zcf. I don’t want to do any manual intervention.

*ALso, I want to exclude some directories.

Solution

This actually took me much longer than it should have. Too many articles insisted you should change the internal field separator (IFS) to a newline character and just use ls. Now in my training I was learn-ed that messing with IFS can be dangerous, and will disrupt many things, including any command that relies on options or arguments (i.e. ALL)
So rather than make my entire script confirm to pesky newlines returned by ls, I did one simpler – eliminate the newlines in place of a decent token separator, like a space!

To exclude directories I just use a magic file “.DONT_BACKUP” that the script checks for.

Code

#!/bin/bash
#
# Backup all directories within webroot
# use empty file ".DONT_BACKUP" to exclude any directory
 
 
# days to retain backup. Used by recycler script
DEFRETAIN=14
LOGFILE=/home/webb_e/site_backups/WebrootBackup.log
#
#
BU_FILE_COUNT=0
#
# and name of backup source subfolder under the users home
WEBDIR=webroot
#
# and name of dest folder for tar files
DESDIR=site_backups
 
#alright, thats it for config, the rest is script
#########################################
 
 
cd ${HOME}/${WEBDIR}/
 
 
TODAY=`date`
BU_FILE_COUNT=0
suffix=$(date +%m-%d-%Y)
printf "\n\n********************************************\n\tSite Backup r Log for:\n\t" | tee -a $LOGFILE
echo $TODAY | tee -a $LOGFILE
printf "********************************************\n" $TODAY | tee -a $LOGFILE
echo "see ${LOGFILE} for details"
 
#for DIR in $(ls | grep ^[a-z.]*$) 
 
for DIR in $(ls | grep ^[a-z.]*$) 
do
	echo $DIR
	#tar the current directory
	if [ -f $DIR/.DONT_BACKUP ]
	then
 
		printf "\tSKIPPING $DIR as it contains ignore file\n" | tee -a $LOGFILE
 
	else
		cpath=${HOME}/${DESDIR}/${DIR}
		#
		#check if we need to make path
		#
		if [ -d $cpath ]
		then
			# direcotry exists, we're good to continue
			filler="umin"
		else
			echo Creating $cpath
			mkdir -p $cpath
			echo $DEF_RETAIN > $cpath/.RETAIN_RULE
		fi
		#
 
		tar -zcf ${HOME}/${DESDIR}/${DIR}/${DIR}_$suffix.tar.gz ./$DIR
		BU_FILE_COUNT=$(( $BU_FILE_COUNT + 1 ))
	fi
 
done
printf "\n\n********************************************\n" | tee -a $LOGFILE
echo $BU_FILE_COUNT sites were backed up
printf "********************************************\n" $TODAY | tee -a $LOGFILE

Result

$ ~/scripts/file_backups/SiteBackup.sh
see /home/myself/site_backups/WebrootBackup.log for details

********************************************
        Site Backup r Log for:
        Thu Apr 15 14:31:53 PDT 2010
********************************************
albums.stonybrookpottery.com
baungenjar.com
bythebootstrap.com
crm.webbmaster.org
digbiz.webroot.com
dotproject.webbmaster.org
edwardawebb.com
kicksareforribs.com
mantis.webbmaster.org
        SKIPPING mantis.webbmaster.org as it contains ignore file
ollitech.com
openx.webbmaster.org
piwik.baungenjar.com
rendinaro.com
riversidegrillnh.com
sidi.webbmaster.org
status.webbmaster.org
steadfastsites.com
stonybrookpottery.com
survey.webbmaster.org
svn.webbmaster.org
taskfreak.webbmaster.org
test.kicksareforribs.com
texpat.webbmaster.org
top.webbmaster.org
        Creating /home/myself/site_backups/top.webbmaster.org
trac.webbmaster.org
trippymedia.com
wave.webbmaster.org
webbmaster.org
wishlist.webbmaster.org

********************************************
33 sites were backed up
********************************************

But wait! We wanted to automate this whole thing right? And so we shall.

Using Cron to automate the process

If your using a webhost they likely provide a GUI to add cron jobs. If that’s the case you can just point to the full path where you saved the above script, select the interval and your good to go.

If your using this on your own server you’ll need to get your hands dirty with a crontab. You can open the crontab file in your editor of choice, or call it from the command line. IN this example we’ll rely in vi, my systems default editor.
Create a crontab file if it does not already exist and open it for edit

crontab -e

You may see some existing lines or you may not. Just remember one job per line. THe layout may seem overwhelming at first, but its quite simple, and breaks down like this

min     hour     day    month    weekday     job_to_Run

The values are in the respective ranges for day of week 0 is Sunday.

0-59    0-23     1-31    1-12     0-6        filename

To omit a field replace it with an asterisk (*) which means all values. Alternately you may use comma separated lists. Although I believe it will treat any whitespace as a delimiter I use tabs to make the organization a little nicer.

So let’s suppose I want to run this job nightly, it is after all named DAILY sql backup
I will add the following line to my crontab

15    0     *    *   *     $HOME/scripts/file_backup.sh > logfile.log

This means every day @ 00:15 a.k.a 15 minutes past midnight it will run the script and print any output into the specified logfile.
If you leave off the redirect to logfile it will email the user with the results. To omit any output use the handy standby

>/dev/null 2>&1

More Help

If you are curious about the script that will actually recycle old backups, then I suggest Log Recycler Script which could easily be updated to handle .tar.gz files instead of .sql.gz files

If you want to backup your MySQL databases, I have that too, Shell script to backup multiple databases

XML vs JSON

Eddie — Sat, 27 Mar 2010 16:20:58 +0000

I recently read an article on Linux Magazine that suggested JSON is a better tool for data transfer than XML on Android phones. So I decided to try switching some of QueueMan’s queue retrieves over to JSON. Eliminating the verbose tags may be much quicker over wireless airwaves.

The choir I tackled was to compare the performance of XML and JSON by downloading the user’s recommendations from the Netflix API. To make testing quick I would make a call to both XML and JSON classes in each loop, and repeat 10-30 times for each sample point. The sample points were 1,5,10,15,25,50, and 64 discs. (64 being the current # of recommendations Netflix is offering me)

I used org.xml.sax, and org.json. JSON* as the libraries for my parsing.

Why JSON is thought to be superior to XML

So the thought, as I have seen and heard it is as follows:

JSON’s lightweight payload allows for reduced bandwidth needs and faster transfers.

That is particularly attractive for mobile devices.

What my Testing Revealed about XML vs JSON

You can see from the 108 samples above that in the 1-5 disc range that JSON indeed performed a bit quicker than XML. However the numbers quickly turned, and JSON grew exponentially slower as the disc count increased. XML’s time grew much slower.

Why XML is a better Performer

The argument that JSON’s has a lighter data definition structure is true, but only by 50%. And the structure for 5 discs is less than 1% of the total character payload (0.8% in JSONs case, assuming an average label of 12 characters, not counting all the commas, parentheses, and escape characters needed). And this drops as the numbers increase. So if you’re just passing a “tag_names” with payloads like “8974″ than JSON would be much lighter. Once you start passing text that far exceeds your tag lengths, than JSON loses its edge.

Size Matters

{ "average_tag" :  {
"term1": "it's all about the data",
"term2": "and the amount of information",
"term3": "you need to transport"
}


"it's all about the data
and the amount of information
you need to transport

So although the payload is just slightly lighter, it has one major trade-off. XML is a parse-once-and-you’re-done deal. The entire XML file is walked once by Sax, and you just build a neat little Object as you go. With JSON you rely on setting objects to represent the structure of the file and pull pieces using their index. I suspect the JSON parsers walk though the string numerous times. (hence the requirement for a string, and not an input stream like Sax.)

So, as far as QueueMan and I are concerned, we’ll stick with XML.

Performance Tip: Boost Your SAX Life

Eddie — Fri, 16 Oct 2009 15:55:23 +0000

SAX, or Simple API for XML has grown to be the standard in Java XML parsing. Numerous studies show how its performance, in the majority of circumstances, beats others such as PULL.

Does that mean you should just bang out a custom handler and expect solid performance, hardly.
This article has only two tips, but it can have a huge impact.

Sample Code

In case it has been a while since you wrote your last SAX XML handler, below is a sample method for the start of each element in an XML stream. This method will be called each and every time a new element starts, no matter how nested or trivial.

public void startElement(String uri, String name, String qName,	Attributes atts) {
		if (name.trim().equals("category")){
			inCategory = true;
		}else if (name.trim().equals("availability")){
			inAvailability = true;
		}else if (name.trim().equals("delivery_formats")){
			inFormats = true;
		}else if (name.trim().equals("queue_item")){
			inQueueItem = true;
		}else if (name.trim().equals("id")){
			inId = true;
		}else if (name.trim().equals("average_rating")){
			inRating = true;
		}else if (name.trim().equals("position")){
			inPosition = true;
		}else if (name.trim().equals("title")){
			inTitle = true;
		}else if (name.trim().equals("box_art")){
			inBoxArt = true;	
		}else if (name.trim().equals("etag")){
			inETag = true;			
		}else if (name.trim().equals("number_of_results")){
			inResultsTotal = true;
		}else if (name.trim().equals("results_per_page")){
			inResultsPerPage = true;
		}
	}

(That is part of a handler I use for Netflix’s API responses.)

Keep it Simple SAX

So this may seem obvious to some of you, and is based on a well understood matter. String comparisons are not cheap. They’re not explicitly expensive, and having a half a dozen in one method would not usually cause any performance bottle necks.

But now imagine calling the same method 1,000′s of times. That’s 6,000 string comparisons!

Each method in a SAX parse will do just that! So how do we reduce the number of comparisons needed?

So if your not using “reasults_per_page” then there is no need to check for that element. It may be useful to create separate handlers for various needs. One can get the quick and dirty summary, while the other can capture all the gritty details.

Hooray! we just knocked off 1,000 comparisons!

Order is Everything!

So keeping the If Else blocks short makes sense, the less comparisons, the faster the parsing. But even for those blocks that require numerous element checks we can tune performance substantially by adjusting the order.

The next tip, and a hugely important one, is to carefully consider the frequency of each element called. To illustrate is a sample XML response form Netflix.

 version="1.0" standalone="yes"?>
 
>
	>26049040137>
	>http://api.netflix.com/users/se3cret/queues/disc?{-join|&|sort|start_index|max_results}
	>
	>32>
	>0>
	>25>
	>
		>http://api.netflix.com/users/secret/queues/disc/available/1/70056440
		>
		>1>
		
			href="http://api.netflix.com/users/secret/queues/disc/available"
			rel="http://schemas.netflix.com/queues.available" title="available queue">>
		 scheme="http://api.netflix.com/categories/queue_availability"
			label="available now" term="available now">Available Now>
		>1185814282>
		 scheme="http://api.netflix.com/categories/title_formats"
			label="DVD" term="DVD">>
		 href="http://api.netflix.com/catalog/titles/movies/70056440"
			rel="http://schemas.netflix.com/catalog/title" title="300">>
		 short="300" regular="300">>
		 small="http://cdn-0.nflximg.com/us/boxshots/tiny/70056440.jpg"
			medium="http://cdn-0.nflximg.com/us/boxshots/small/70056440.jpg"
			large="http://cdn-0.nflximg.com/us/boxshots/large/70056440.jpg">>
		 href="http://api.netflix.com/catalog/titles/movies/70056440/synopsis"
			rel="http://schemas.netflix.com/catalog/titles/synopsis" title="synopsis">
			>Zack Snyder directs this faithful adaptation of Frank Miller's (Sin City) graphic novel about the storied Battle of Thermopylae, a conflict that pitted the ancient Greeks against the Persians in 480 B.C. The film, which blends live-action shots with virtual backgrounds to capture Miller's original vision, co-stars Gerard Butler as the Spartan King Leonidas, who leads his small band of 300 soldiers against an army of more than one million. ]]>>
		>
		
			href="http://api.netflix.com/catalog/titles/movies/70056440/bonus_materials"
			rel="http://schemas.netflix.com/catalog/titles/bonus_materials"
			title="bonus materials">>
		>2007>
		 scheme="http://api.netflix.com/categories/mpaa_ratings"
			label="R" term="R">>
		 scheme="http://api.netflix.com/categories/genres"
			label="Action & Adventure" term="Action & Adventure">>
		 scheme="http://api.netflix.com/categories/genres"
			label="Adventures" term="Adventures">>
		 scheme="http://api.netflix.com/categories/genres"
			label="Military & War Action" term="Military & War Action">>
		 scheme="http://api.netflix.com/categories/genres"
			label="Epics" term="Epics">>
		 scheme="http://api.netflix.com/categories/genres"
			label="Comic Books and Superheroes" term="Comic Books and Superheroes">>
		 scheme="http://api.netflix.com/categories/genres"
			label="Blu-ray" term="Blu-ray">>
		 scheme="http://api.netflix.com/categories/genres"
			label="Warner Home Video" term="Warner Home Video">>
		>6960>
		
			href="http://api.netflix.com/catalog/titles/movies/70056440/format_availability"
			rel="http://schemas.netflix.com/catalog/titles/format_availability"
			title="formats">
			>
				 available_from="1185840000">
					 scheme="http://api.netflix.com/categories/title_formats"
						label="Blu-ray" term="Blu-ray">>
				>
				 available_from="1185840000">
					 scheme="http://api.netflix.com/categories/title_formats"
						label="DVD" term="DVD">>
				>
			>
		>
		
			href="http://api.netflix.com/catalog/titles/movies/70056440/screen_formats"
			rel="http://schemas.netflix.com/catalog/titles/screen_formats" title="screen formats">>
		 href="http://api.netflix.com/catalog/titles/movies/70056440/cast"
			rel="http://schemas.netflix.com/catalog/people.cast" title="cast">>
		
			href="http://api.netflix.com/catalog/titles/movies/70056440/directors"
			rel="http://schemas.netflix.com/catalog/people.directors" title="directors">>
		
			href="http://api.netflix.com/catalog/titles/movies/70056440/languages_and_audio"
			rel="http://schemas.netflix.com/catalog/titles/languages_and_audio"
			title="languages and audio">>
		>4.0>
		 href="http://api.netflix.com/catalog/titles/movies/70056440/similars"
			rel="http://schemas.netflix.com/catalog/titles.similars" title="similars">>
		 href="http://www.netflix.com/Movie/300/70056440" rel="alternate"
			title="web page">>
 
	>
	>
		...
	>
 
>

So there are some important things to note;

‘queue’ occurs once, and only once
Likewise ‘etag’, and result counts occur once and only once
‘queue_item’ occurs once for each movie (1 to many times)
‘id’, ‘position’ and movie details also occur once for each listing (1 to many times)
‘availability’ may occur 1 to 3 times for each movie (3 to (3*many) times )
‘category’ is by far the most notorious, occurring within each title multiple times, and within each availability (lots of times)

So what does that mean? It means the very first element we’ll check for is ‘category’, then ‘availability’, and work our way to the least common element, ‘queue’.

This is a very effective way to increase performance because once a match is found, we skip all the rest.

So a 12 comparison method effectively becomes a single comparison if we are in a ‘category’, and a 2 comparison method if we are in ‘availability’, etc. Until finally on the rarest ‘queue’ which occurs only once do we evaluate all 12 comparisons.

Results

This test is hardly conclusive, but you can see the average and max parse times of a randomly arranged handler and a popularity arranged handler.

Automatically subscribe users to DreamHost announce lists

Eddie — Wed, 19 Aug 2009 18:58:01 +0000

This is a response to a question on the DreamHost wiki posted by anonymous.

“I have a Contact page using form mail, and want to include a checkbox that enable visitors, to also subscribe to our Announce List when posting their form mail.

Is there a facility for adding users to the Announce List without using form POST”

Without using POST? I am not sure about that.. but using a checkbox to subscribe users is a snap.

Your resources;

DreamHost Panel API
PHP curl methods

So it goes like this;

Your user registers a new account, or sends you a contact message. As part of the form they submit their email address and name. We in turn pass that email and name onto DreamHost’s panel API thereby adding the user to future announcements.

The code I will explain needs to go in the form’s receiving code. We will check for the checkbox’s value, and if necessary handle the additional choir. This code should be pasted into your contact form or registration form wherever the initial data is received and handled.

The red boundary denotes new code for the announce submission

Since every cms, site and blog are different, I am unable to provide specifics, but yu should be able to track down the code for your form and find the portion that handles the submission. (Most obviously denoted with $_POST or $_GET variable use.) I will also assume your email field is named ‘email’ and your name field is two fields; ‘firstName’, ‘lastName’;

Details on the API command

Below is a link to details on the command, needed parameters and possible responses.
Add Subscriber API Command

The Announce Submission Code

if(isset($_POST['subscribeMe']) && $_POST['subscribeMe'] == 1)
{
 
       //get the values we need from form(this should in aprt already be somewhere in the code your editing)
       $email=$_POST['email'];
       $fullName=$_POST['firstName']." ".$_POST['lastName'];
 
       //set values we shoudl know, and are constant
       $domain="domain of this form";
       $listname="list-name";
       $apiKey="666666666666";
 
       // using curl and passing 5 critical values to the api
       // list, domain, email, name and API Key
 
 
		$ch = curl_init('https://api.dreamhost.com/');
 		curl_setopt ($ch, CURLOPT_POST, 1);
 		curl_setopt ($ch, CURLOPT_POSTFIELDS, "key=$apiKey&cmd=announcement_list-add_subscriber&listname=$listname&domain=$domain&email=$email&name=$fullName");
 		$result=curl_exec ($ch);
 		curl_close ($ch);
 
		if ( stripos( $result,'success') > 0){
			echo "Congrats! 
You have been added to our Mailing List
";
		}else{
			echo " Ooops! 
Unable to add your email to our announcement list please contact site administrator.
";
			echo "Code: " . $result;
		}
}// end if subscribe box checked

Getting a DreamHost API key

More info on DreamHost API

Getting the most of cell phone favorites – a.k.a. free calls

Eddie — Sun, 12 Jul 2009 14:15:27 +0000

Many cell phone carriers are moving to implement a circle of friends that you can call for free. T-Mobile has MyFaves™, Alltel has My Circle ™, even Verizon offers a discount on the ten numbers you call most. This begs the question;
“Who do I call Most?”
Well you can be subjective about it and decide you like Danny better than Mary-Lou, but the truth is that Mary-Lou calls you constantly, and that’s going to put a dent in your anytime minutes.

The more objective and scientific method is to actually count the minutes you spend on each number and rank the accounts in order of use. Hmm, I am already thinking AWK.

Read on to get the awk script to sort your monthly calling logs into facts you can use!

The Problem

You need to know if your circle of chosen numbers is giving you the best use of your free calling.

The Solution

Tally up those calls and compare. This uses some stanard ‘nix utils including awk and sort.

Start by jumping to your carriers website. They all provide a digital view of your call logs. You can copy and paste into a text file, or download a CSV file that lists them out. Name this file callLogs.

(If you are using a CSV, I recommend replacing the commas with semi-colons, or another character that won’t be found anywhere else in the record. Removing extraneous quotations is helpful as well.)

Sample Call Log

07/11/09;INCOMING;06:09 PM;555-420-1118;;1
07/11/09;PLATTSBG, NJ;05:55 PM;555-420-5141;(V);1
07/11/09;PLATTSBG, NJ;05:55 PM;555-420-1318;;1
07/11/09;INCOMING;05:38 PM;555-420-5141;(V);2
07/11/09;PLATTSBG, NJ;05:20 PM;555-420-5141;(V);1
07/11/09;INCOMING;05:15 PM;555-420-5141;(V);2
07/11/09;VM Retrieval;05:12 PM;221;(F);5
07/11/09;POUGHA, NJ;04:04 PM;555-643-8173;;2
07/11/09;VM Retrieval;04:04 PM;221;(F);1
07/11/09;PLATTSBG, NJ;12:23 PM;555-563-1689;(V);2
07/11/09;INCOMING;12:36 AM;555-593-1910;;2

This is a T-Mobile Call Log. There are 6 fields. I only care about 3 of them. The number ($4) Their bucket, e.g. MyFave, Mobile to Mobile, ($5) and the call time ($6).
Excellent. Now we need some script to sort, total and report on those numbers. Enter AWK.

The AWK Script to Total and Sort Call Logs

*** Please pay attention to the field numbers used ($4,$5,$6) the order and count of your fields may differ ***

# awk script that totals minutes based on verizon call log (pasted form site)
 
#usage: cat callLog | awk -F";" -f  
 
#in T-Mobile world a F denote mobile to mobile and V denote my fave
#will have 3 buckets in total then
$2 !~ "([VF])" {minutes[$1] += $3}  #whenever minutes
$2 == "(V)" {faves[$1] += $3} #myfaves
$2 == "(F)" {tm[$1] += $3} #mobile to mobile
 
#print a nice summary please
END {		
		for (i in minutes)
		{			
			 total += minutes[i]
			printf "%s calltime totaled %d\n", i, minutes[i]
		}
		printf "Total: %d",total
		printf "\n\n Faves Totals\n"
		for (i in faves)
		{			
			 ftotal += faves[i]
			printf "%s calltime totaled %d\n", i, faves[i]
		}
		printf "Total: %d",ftotal
		printf "\n\n T-Mobile Totals\n"
		for (i in tm)
		{			
			 ttotal += tm[i]
			printf "%s calltime totaled %d\n", i, tm[i]
		}
		printf "Total: %d",ttotal
	}

Great, now we can run our call log through the awk script to see what we get..

The Result

I am on T-Mobile, so I have 3 Categories of calls; My anytime or whenever minutes, my MyFaves™ minutes, and my T-Mobile to T-Mobile calls.

Eddie@HP-dv6z-420 ~
$ cat callLog | awk -F";" -f ~/scripts/CallLogSummary
555-448-3547 calltime totaled 2
555-572-7255 calltime totaled 2
555-593-0910 calltime totaled 5
555-643-8773 calltime totaled 2
555-433-1328 calltime totaled 8
555-578-2088 calltime totaled 2
555-682-8397 calltime totaled 2
555-335-1974 calltime totaled 3
555-334-3041 calltime totaled 5
555-373-8408 calltime totaled 7
555-593-2753 calltime totaled 26
555-351-8146 calltime totaled 12
555-420-6768 calltime totaled 4
555-531-0398 calltime totaled 4
555-420-6318 calltime totaled 8
555-448-2571 calltime totaled 1
555-740-4700 calltime totaled 1
Total: 94

 Faves Totals
555-563-0689 calltime totaled 76
555-420-5169 calltime totaled 23
555-593-1531 calltime totaled 5
555-420-5641 calltime totaled 23
530-721-0449 calltime totaled 23
Total: 150

 T-Mobile Totals
123 calltime totaled 12
555-420-8399 calltime totaled 4
221 calltime totaled 6
Total: 22

As you can see I am NOT getting the most of my plan. I will switch the low My Fave out, and add the Highest called number from the first block.

Keep your inbox clean with email aliases

Eddie — Sun, 05 Jul 2009 15:28:28 +0000

More and more sites require you to verify an account by receiving an email.. fair enough. But then 2 months later you start getting the newsletters and updates that you no longer care about.

Enter Gmail !

To be honest, other mail servers may do the same, it may be standard, but I have not seen it used outside of gmail domains. Anyway, I digress.

The Alias ability in Gmail lets you create a unique address for a site(s) and then set filters on those addresses, my favorite being send to trash..

Email Aliases – Best Approach?

Ok so the downside is you can’t turn off the aliases, and senders won’t get a non-recipient error when they send the email. Most intelligent mass mailing systems will remove addresses that get this error on several attempts. So your not doing your part to cut down useless internet traffic.

BUT! you will never see the email and cringe with anger because you still have not bothered to login to the site, close the account and unsubscribe.

The Gmail Way

When you are signing up to a site with obligatory email, give them your email as follows;

my.name+uniquetag@gmail.com

You’ll get the one time “please click here” activation email just as if you provided your normal address.

Then, once you decide you no longer need to read their messages you can set the filter. Just enter a plus sign and a unique tage ( ’+uniquetag’) as the to: criteria. I get a ludicrous amount of emails from this Eddie fellow about cats, so its time I made a filter..

Building a Filter from Gmail Alias

Other Ideas

So there are lots of reasons to use this aside from email address discretions.

Labeling your email forwarded from other accounts, just change the FWD address in the other account
Sorting comments from a website contact form (+feedback, +sales, etc)
Telling people your email address has a plus sign, beat that!
Auto-Archive daily Cron messages

I would love to hear about some more handy Gmail tricks.