Karu Ja Hundu
Software, Electronics, Music, Cooking, Photography and all-round Geekery

Learning node.js with udemy.com

It has been brought to my attention that udemy.com are now offering an extensive, free node.js tutorial. Learning node? Check it out!

banner_Node

By Moshfegh Hamedani for Udemy

2015-10-06 20:25:54

Random (day) date generation in MYSQL

I needed a way to randomly generate a "created at" record date for a test I was working on. The dates only needed to be on different days so the time part of the timestamp wasn't important.

UPDATE `the_table`
SET `created_at` =
(CURRENT_TIMESTAMP - INTERVAL FLOOR(RAND() * 7) DAY)
2015-06-23 09:02:52

PHP Xdebug - can not get property

I've been using the PHP debugger XDebug for several years now and for the most part it's been great. However, in the last two years I've had a lot trouble with acquiring the value of some watched variables. Usually long strings like a SQL query or a large paragraph of text. I initially thought this was a problem with PHPStorm but after looking in to their bug reports I discovered it is a known bug with the debugger itself and not the IDE.

Solution

Get the latest build of XDebug. This can be achieved by cloning the repo from the github.

git clone git://github.com/xdebug/xdebug.git

A great way to get the linux build instruction is to use the "Tailored Installation Instructions" which are provided by using the XDebug install wizard. This wizard is excellent, it requires the text from your phpinfo() or php -i and produces tailored instructions - very handy. To get the text you need run this command;

php -i | xclip && xclip -o > ~/php_output.txt

Now follow the instructions provided to you by the wizard and your done. You're now running the latest build of xDebug which seems to fix the issue.

Things to remember

You're now running the latest, Dev build of XDebug, I don' know if it has been extensively tested so be careful if you're using this at work.

2015-06-22 12:21:26

MySQL Anti-Joins

I was trying to find all entries in one table that no longer existed in another. After a bit of messing about I came up with this query. As it turns out this seems to be referred to as an anti-join. For more information on this have a look at this great post on anti-join performance.

SELECT   a.`id`
FROM  `table_a` a
  LEFT JOIN `table_b` b
    ON b.Hash = a.Hash
WHERE b.Hash IS NULL
2015-05-13 14:20:31

Random selection in MySQL

I needed to randomly select ten records from a database table I was working with. I achieved this by using the RAND() operation with a limit set on the output.

SELECT `idnum` FROM `a_table_to_select_from` ORDER BY RAND() LIMIT 10;

The RAND() function can also be used to delete random records. 

DELETE FROM `a_table_to_select_from` ORDER BY RAND() LIMIT 10;
2014-08-13 11:37:16

Restart Gnome3 shell

Gnome3 crashes a lot, maybe once a week. This probably due to the mix of software I'm using. I have noticed that Hipchat shows up in the Gnome shell error log, something to do with an invalid window resize - oh well. When Gnome would crash I would normally restart the computer however I've recently learned a new trick - you can restart the gnome shell!

Select a new virtual console 

Ctrl-Alt-F1

Restart Gnome Shell

gnome-shell --display :0 --replace

Navigate back to the crashed shell

Ctrl-Alt-F7
2014-08-11 09:52:34

Hotyop update!

I spend a little while this evening updating the hotyop project. The biggest update is the addition of a charts section which provides some metrics based on the data gathered over the last few months. The plan is to write some SQL which will provide more interesting information rather than the basics like min, max and average groupings. I also updated the JS callback api so it's a tad bit faster now.

hotyop.com

2014-07-10 21:04:19

SQL joins revision

I was looking for a good resource to help remind me how each of the different SQL Joins worked. I found this excellent imagine on codeproject.com

 

SQL Join Cheat Sheet

2014-07-02 10:15:02

A CSS only way to create tooltips

I was looking for a way to deliver more information to the user without overloading the UI with possibly unnecessary text. This website illustrates a way of using the data attribute with CSS to create tooltips without the need to us JavaScript.

2014-04-18 08:34:16

Code profiling and profiling tools

Code profiling is a great way to help identify bottlenecks in your code. I know this because I recently spent some time learning how to profile my code using xdebug profiler and kcachegrind.

I've been building a new feature for my company's product for the past five months. It's after getting very big, not just code-wise but in terms of its reach throughout the application. Because of this, making sure the feature runs as fast as possible has become very important.

After a few short steps I had the profiler configured to write its logs to the /tmp directory. I started the profiler and ran through a standard use cases of my feature. The profiler creates a lot of files, which aren't very informative using something like a standard text editor. To actually make use of these large log files you will need to use a profile parsing utility of some sort. These tools take the data logs gathered by the profiler and turn it in to meaningful information.

I chose to use kcachegrid as it seemed to be widely used.  I was able to view metrics on the most called functions and the functions which took the longest to execute. I then used this information to see if I could do anything to my code to improve execution time. As it turned out the function that was taking a long time to execute was a Model function which was querying the database. I had a look at the query being called and discovered that not all the columns being used in table joins were indexed. I quickly fixed this and reran my use case. The slow function call in my use case when from taking 5.33 seconds to only .17 seconds; A massive improvement. 

I fully expect I will continue to use code profiling to help identify problem areas in the applications I work with. I really only used it to help identify slow functions, but I imagine it would be very useful for keeping metrics on system performance on a per-release basis. Or, help identify areas of code that could do with some work.

I used this excellent article to help get the profiling extension running. It also goes in to the different tools you can us to parse the profilers output - check it out.

2014-04-11 09:51:14

Post comments with Disqus

I finally got around to adding post comments. I decided to go with Disqus because it looked the easiest to install and moderate. After registering I was given a snippet of HTML and JS to put into my post view. Simple.

disqus

2014-03-26 20:45:10

Working on the wrong GIT branch

I discovered I was working on the wrong GIT branch today. My initial taught was to copy the code changes in to a text file and then checkout the feature branch I was supposed to be working on. After a second or two I realised that was fairly stupid and I decided to use stash.

First I stashed the changes I made on my current checked out branch.

git stash

Next I checked out the branch I was supposed to be working on.

git checkout feature_branch

Lastly, I applied the stashed changes to the checked out branch.

git stash apply

I could have also used git stash pop. This would have removed the stashed code changes from the stash.

2014-03-19 12:00:55

CSS Bug

I've discovered a style issue with the contact and login buttons. The login button seems to pop out of the menu bar div and I've no idea how to replicate it. So far it's happening on my laptop only. If you you see the issue you might fix it for me - you can email me the CSS fix, I'd be very grateful :D

2014-03-17 21:43:32

Hotyop Update

Hotyop has been updated!

Look

The site looks better (or at least different)! It now behaves more like an application. I've achieved this feel by making Hotyop act more like a One Page Application

I've added a menu bar to the top of the application. Two modes are currently available. You can either view the most recent recorded data or view a visualisation of all recorded temperature and humidity data. These pages are loaded by injecting HTML and Javascript into the application's main container.

Engine

I've completely changed how the site works. I created an API, which sites on a sub-domain hosted by a Raspberry Pi. This API returns different Javascript callback functions which Hotyop uses to acquire current reading data and data for visualisation?

2014-03-09 21:35:15

GIT, Premature Commit

I hate it when you commit some code only to discover you've forgotten to remove a test var or comment. This GIT command rolls back the last commit without loosing your code changes. You can then make the necessary changes to your branch and commit again.

git reset --soft 'HEAD^'
2014-03-07 14:40:08

SSH Socks Proxy

I don't always trust the network I'm connected to or I can't access a website or web service I need because of a firewall restriction. A quick way around this is to tunnel your traffic to a server over a encrypted connection. 

ssh -D 8080 -f -C -q -N user@myserver.net

Specifies a local “dynamic” application-level port forwarding - in this case port 8080

-D 8080

Requests ssh to go to background just before command execution

-f

Requests compression of all data 

-C

Quiet mode. Causes most warning and diagnostic messages to be suppressed.

-q

Requests compression of all data 

-N
2014-03-05 09:36:01

Finding words in files

I was looking for a Python script today but I couldn't remember where it was stored or what it was called. I knew the username for a database connection the script was using so I employed grep to help me find the file which contained that username. Simple.

grep -r "string" . 
2013-11-19 10:37:46

MySQL - Create Statements

I was looking for a way to extrapolate the create command for a SQL View I had build a year ago. As it turns out MySQL has a function for it already;

SHOW CREATE VIEW 'some_cool_view'';
2013-09-24 11:39:37

Programmers' Day

I'm spending my day desperately trying to get a feature coded up before the senior Dev in charge of the project I'm working on comes back from holidays. I wonder if he'll mind if I skip data validation?

2013-09-13 10:38:57

Raspberry PI overclock and benchmark

I was looking for a way to test the performance of my Raspberry Pi. I wanted to see how much of a difference there was between running the CPU at 700MHz and 900MHz. As it turns out, overclocking the Pi's CPU to 900MHz provided it with a 23.40% boost in my tests. For the test I used a CLI program called sysbench.

sysbench --test=cpu --cpu-max-prime=1000 run
2013-09-12 15:59:30

External to Inline css E-mail styling

I discovered today that many email clients don't like external CSS. As a result of this I spent a while rewriting the styling for a HTML based email so it looked the same on 'most' email clients. After doing this work I then discovered this site. I wish I had found this sooner.

2013-08-22 14:53:40

MySQL - More to learn

It finally happened. I had to learn how to script in MySQL. I decided to spend some time reading about stored procedures before I got stuck in and I found this website really useful. 

I needed to do a good bit of string manipulation to one of the columns of a table I'm working with. I should also point out that I didn't have access to a scripting language. I was forced to do all this string manipulation in SQL.

I ended up writing a SQL function and used temporary tables for the first time also.  

 

2013-08-21 16:18:15

Get the top 25 largest files

This is a really nice shortcut/script that I recently added to my 'usr/local/bin' dir.

File: diskUsage.sh
#!/bin/bash
du -hsx * | sort -rh | head -25

When executed the script returns the top 25 largest directories / files from your current path.

1.3M    apache2
268K    bootstrap.log
164K    auth.log.1
108K    ConsoleKit
100K    daemon.log.1
80K    apt
52K    syslog
48K    wtmp.1
48K    wtmp
48K    dpkg.log.5.gz
40K    auth.log
36K    syslog.1
36K    daemon.log
32K    dpkg.log.2.gz
20K    messages
20K    kern.log
20K    auth.log.3.gz
16K    Xorg.0.log.old
16K    Xorg.0.log
16K    dpkg.log.1
16K    auth.log.4.gz
16K    auth.log.2.gz
12K    lastlog
12K    fsck
12K    dmesg.0
2013-08-08 11:32:24

MySQL Views - another good reason for

I like using views. They help reduce the dependency between your software model layer and the database by placing a representation layer between them.

I friend of mine is working on building reports using a proprietary query language. He's had many issues trying to manipulate the data because the query language he's using doesn't support the functions he's come to expect, concat for example.

His solution was to create a SQL view which would have all the data pre processed before it got passed to the reporting engine. 

2013-07-04 12:42:41

Node.js learning resource

I was looking for some info on node.js and found this.

2013-06-24 16:56:30

Java Update and Version Switching

Check out this very useful link if you're interested in learning how to switch the Java version currently running on your server.

Java Swap

2013-06-20 15:55:36

Things to consider, UI Design

In college I did a few modules on UI design. Two of the resources I still use today when examining / building a user interface are;

10 Usability Heuristics for User Interface Design

Shneiderman's "Eight Golden Rules of Interface Design"

2013-06-18 23:42:29

Bye Bye OSX, Hello Debian

About four years ago I moved from Windows to Mac OSX. Since leaving college I also found the companies I've worked for also used Mac OSX.

A constant over these years has been the presence of Linux in all my server tasks, personal and professional. I found myself spending more and more time working with Linux, so much time in fact that I started to miss it when I returned to OSX to do my development work.

Today I made the move and installed debian on my MacbookPro and PC. 

debian logo

2013-06-15 19:16:29

Raspberry Pi Temperature

I recently discovered you can get the core temperature of the Raspberry Pi;

 

cat /sys/class/thermal/thermal_zone0/temp

2013-06-10 12:03:40

Karu has moved to Pi

Today I moved my website from my house server to my Raspberry Pi.

The server cost €106.93 a year to run while the Pi will only cost €7.15.

I've noticed the the website is a little slower but all in all I'm happy with the website performance.

2013-06-09 19:22:54

Keeping your Linux distro up-to-date

The following command is a nice and easy way for you to keep all your OS packages up-to-date

sudo apt-get update && time sudo apt-get dist-upgrade -y
2013-06-08 09:37:55

GIT: Deleting a remote branch

I make a lot of branches. They allow me to work on several problems independently from each other. I often push a branch I'm working on to a remote repo and get other developers to look at say, a bug I'm trying to fix. Doing this is great but if you don't clean up temporary branches that don't really have a place on the remote repo people (your boss) might get annoyed with you. This admin work is easy;

git push [remote] :[branch]
e.g. git push origin :bug_IG53 
2013-06-05 10:39:01

Life...

try{ Life::throw_obstacle(); }catch{ continue; } // just get on with it
2013-05-28 15:05:17

Directory size

I recently wanted to compare two directories to see if they were the same size. To get the size of the directories I was able to use this lovely little command.

du -sh ~/
2013-05-20 10:38:10

Find is great when setting up new sites

I recently learned you can use find in conjunction with other commands. Here is an example of this; I'm applying some file permissions and changing the file owner based on fthe group these files belong to - in this case root.

find /var/www/your.website.com -group root -exec chmod 755 {} +
find /var/www/your.website.com -group root -exec chown apache_user:www-data {} +
2013-05-17 09:53:38

Finding things with 'find'

I was using a distro of Linux I wasn't familiar with today and couldn't find the httpd.conf file. I was able to get the OS to find it for me using find. The / parameter specifies the scope of the search. In this case I wanted to search the entire machine.

 

find / -name 'httpd.conf' -print

 

By using the parameter . we can tell find to search only the current directory.

 

find . -name picture_library

2013-04-25 15:11:51

Add New RSA Identity

When working with tools like GIT you'll eventually find you quickly grow tired of entering your repos credentials over and over again. You can remedy this by setting up your repors to allow SSH.

Generate new RSA Key for the new service you want to authenticate with.

Add the newly generated RSA private key to your ssh agent. In this example I'm adding my RSA private key bitbucket to my ssh agent.

I often get paranoid that I've copied some extra white space or missed a character when copying the public key into my keys list on the GIT repos admin client. Use this command to copy the entire content of the public key into your clipboard.

Depending on the git repos provider you are using. Or, whatever you are using that requires ssh auth there are many different ways to add your new public key. For bitbucket the option is found under Account and Keys. Simply paste in the new key and your done. 

If you already checked out an old repository using HTTPS you will need to update the protocol this repository uses to communicate to remote. This can be done by going to the repository and editing it's config file. 

[remote "bitbucket"]
    url = git@[USER].bitbucket.com:/[REPOS].git
    fetch = +refs/heads/*:refs/remotes/bitbucket/*

2013-04-09 12:23:23

Raspberry Pi Video to HDMI pusher

I created this program to help headless PI users to push video and sound through the HDMI interface.

GIT Repo Here

Screenshot of push to screen app

2013-03-24 02:06:48

Room Occupancy - Visualisation Engine V1.1

I spent a while last night getting a new data chart in place. I really wanted to visualise the entire database so I spent some time optimising the MySQL view and have managed to provide an accurate view of the room entire temperature and humidity readings.

This new view is available here [click].

 

I should also point out...

I'm using Google's annotated time line chart for the visualisation. The temperature sensor used in this project is the DHT22 (data sheet available [here]) and it was kindly given to me by this smart fella.

2012-12-11 08:37:33

Room Occupancy - Visualisation Engine V1.0

I've been working on a visualisation engine to take data stored in MySQL views and present this data in a multitude of ways. Currently I have the basics in place. The MySQL views work and the graph engine can power one type of visual. You can access the beta link now at [here].

2012-12-10 18:43:56

MySQL: Views, Events and Procedures - Fun!

As part of my room occupancy project I decided I wanted to keep track of the percentage change in room temperature. I plan on doing some fun stuff with this data once I get the foundations for the system in place. For now I plan on using the data I will use it as a flag - allowing me to improve efficacy and also prevent my system from running wild and waisting electricity (just in case the temperature and humidity sensor was to burn out).

I achieved this data gathering by creating three components. A view to aggregate the average temperature over the last ten minuets and calculate the percentage change based on the current live reading. A procedure to select this data from the view and store it in an archive table for later calculation and finally an event. The event was used to call the stored procedure on schedule basis. 

View

SELECT
  avg(`a`.`temperature`) AS `average_temp`,
  `b`.`temperature` AS `current_temp`,
  (`b`.`temperature` - avg(`a`.`temperature`))AS `difference`,
  IF(((`b`.`temperature` - avg(`a`.`temperature`)) >= 0),
     'Positive',
     'Negitive'
  ) AS `Change`,
  (((`b`.`temperature` - avg(`a`.`temperature`)) / avg(`a`.`temperature`)) * 100)
    AS `percentage_difference`
FROM
  (`view_last_10_readings` `a` JOIN `view_current_data` `b`)

Procedure

DROP PROCEDURE IF EXISTS `StoreDifference`;
DELIMITER ;;
CREATE DEFINER =`webservice`@`localhost` PROCEDURE `StoreDifference`()
  BEGIN
    INSERT INTO webservice_difference (
      `average_temp`,
      `current_temp`,
      `difference`,
      `change`,
      `percentage_difference`,
      stamp
    )
      SELECT
        *, NOW()
      FROM
        view_percentage_difference;
  END
;;
DELIMITER ;

View

DROP EVENT IF EXISTS `Automation`;
DELIMITER ;;
CREATE EVENT `Automation`
  ON SCHEDULE EVERY 10 MINUTE STARTS NOW()
  ON COMPLETION NOT PRESERVE ENABLE DO CALL StoreDifference()
;;
DELIMITER ;
2012-11-24 12:12:11

Room Occupancy: Comfort For Less

Background

With cold days rapidly approaching I decided to buy an electric heater for my apartment. My apartment relies on old storage heaters which run at a minimum of 2kWh. If I leave one of these guys running over night that's approx. 14 units of electricity. Going by my latest 'Electric Ireland' bill a standard NightSaver unit (1 kWh) is 10.18 cent. So, to run the heating for one day of warmth it will cost 142.52 cent (Remember the heaters are rated at 2 kWh). Using the electric heater which is rated at 800w I can heat my living room when I come home from work until I got to bed. This is usually about 5 hours during what 'Electric Ireland' deem as Standard Domestic time - A unit (1 kWh) at this rate costs considerably more - 19.28 cent. Remember my heater is rated at 800 W, that's 20% less than the standard domestic unit. So, I'm effectively being charged 15.424 cent an hour. For those 5 hours that I'm at home it costs me 77.12 cent to heat the living room - that's almost half the price.

Objective

Since buying the electric heater I've enjoyed many a warm evening, I can't complain. However, I find myself in conflict. Every time I turn the heater on I'm very aware that I'm spending my hard-earned money. I've decided to try to make the electric heater intelligent so that it is aware of two factors, the rooms current temperature and humidity over the course of the day and what I regards as comfortable. Using these two determinants the heater will then regulate the rooms temperature so that it is within a threshold defined by my idea of comfortable.

Device Learning

I want the electric heater to learn two things. It needs to learn what I believe is a comfortable room temperature and it also needs to learn how the temperature fluctuates throughout the day. My belief is that by learning when the temperature of the room naturally drops and rises I can use this data to more efficiently achieve a desirable level of room comfortability. The desired result being a warmer living room at a lower cost. 

Division Of Work

This project will be divided into three parts. The first encompassing the design of a web service which will retrieve data from a room sensor. The room sensor will record temperature and humidity data and send it at regular intervals to the web service. 

The second part of the project will consist of the wiring up of the room sensor to the electric heater so that it can be controlled. This will be achieved using either a digitally controlled relay or if the heater permits a simple digital switch to turn the heater on and off.

The final part of the project will consist of programming the logic behind the intelligent room thermostat.

2012-11-18 13:17:02

Recursion

I was faced with a problem at work the other day that was great fun to resolve. I needed to navigate a directory structure and store all files of type .doc and .pdf in an indexed array structure. I realised that only two things needed to be checked; is the directory element a director file or a .doc / .pdf file. I solved this problem by writing a recursive function. Simply put the function calls itself an indeterminate number of times until the entire directory is parsed and all files are accounted for.

public static function document_table_builder($doc_array, $documents = null,$counter = 0, $path = null){
      foreach ($doc_array as $key => $value)
      {
      if ($value['item_type'] == 'D')
      {
         $documents[$counter] = array('id' => $value['id'], 'name' => $value['title'], 'path' => $path, 'created' =>       $value['created_date'], 'last modified' => $value['modified_date'], 'size' => number_format($value['filesize'] / 1024, 2, '.', ''), 'type' => $value['mime_display'], 'download' => self::create_download_link($value['id'], $value['title']));
        $counter++;
      }
      else if ($value['item_type'] == 'F' && !empty($value['items']))
      {
         $path = $value['full_path'];
         $rec = self::document_table_builder($value['items'], $documents, $counter, $path);
         $documents = $rec['document'];
         $counter = $rec['counter'];
      }
   }
   $rec = array('document' => $documents, 'counter' => $counter);
   return $rec;
}
2012-09-20 15:26:53

How to make yourself feel at home, while at work.

I'm a software developer. This means I am constantly entering data into the software I make to test that it actually works. I recently discovered that by using the names of your family members it can actually help improve your mood while in the office. Now while I sit at my desk, typing out lines of code, I hear people talk about the systems functionality by using the test data I entered - my family.

"You see, Emma has made two transactions today and she has two outstanding transactions from last month"

Oh Emma, You and your transactions.

2012-09-18 14:58:04

Find and Replace with 'sed'

I find I use the UNIX command sed a lot. Today I had to do some modifications to a clients websites. I dumped the pages of the site from its MySQL database and was able to use sed to make the necessary changes without having to go near our CMS. Here I used sed to find =" and replace it with =". I also removed and changed some recurring text.

sed 's/=\"/="/g' ./all_pages.sql > formatted_pages.sql
2012-08-20 17:30:37

AWK to the rescue

I was asked to produce a list of all the sites currently being hosted on our companies web server. ls -l looked messy so I decided to give awk a try. I was able to use awk to pull the ls -l output apart into vars and then choose the ones I wanted for the report. Much better :)

ls -l | awk  '{print $9, $10,  $11}' > /home/david/test_sites.txt 
2012-08-07 14:40:04

LEFT, RIGHT JOINS

I found myself developing a bit of a bad SQL habit recently. I was avoiding JOINs and instead I was trying to do everything with a mess of nested selects - sometimes 3/5 selects deep. I decided that today I would relearn my JOINs - Jesus they makes things so much easier. I first covered these in 1st year of college. Why did I every fall out with them?

This site gives a great breakdown of the different JOINs available.

2012-07-24 10:20:16

Karujahundu now has authentication

I finally go around to it and set up user, and user roll authentication for this blog engine. I had started the process a few weeks ago but never got around to doing it. 

As with most things (except for databases manipulation) related to Kohana, adding website authentication was simple - much easier to implement than the authentication class I wrote in my 4 year of college anyway. I found this good blog post which really helped work out what Kohana's Auth class was doing.

Oh and yes, all the admin functionality was publicly available up until this point, I had Google's spider delete a few posts on me at the beginning :)  

2012-05-21 23:36:12

Learning Java?

In college I took a module on Web Services Development using OWL, BPL, SOAP, WSDL and all those fancy things. This was a really cool module but unfortunately my knowledge of Java limited my creativity in the subject. My lecture suggested I got a book called Head First Java. I did and in a few short weeks I had completed the book and, more importantly, I had hugely extended my knowledge. 

I really recommend this book. It's not just a book on Java, it's a book on Object Oriented programming too (which is nice).

Get it here.

2012-05-06 10:52:12

Starting with iOS

I decided to learn how to program for iOS. I've already made a start with Android but as I've recently moved to the iPhone I decided I'd also change my mobile app dev focus. I came across this tutorial today and I think it's a great place for anyone interested in iOS dev to start: Click and Learn

2012-04-15 22:12:10

I hate isset()

Today I spent far too long typing is_set() into my IDE. I couldn't for the life of me work out why the function wouldn't work anymore. Had it been depreciated over night somehow! I eventually gave up and searched the Internet. I've used this function a million times in the past and not once did I notice that it doesn't follow the normal naming convention of similar functions like is_array() or is_numeric(). Silly me!

2012-04-09 15:50:43

Photo: IdeaBubble lads

This rather poignant photo (well, as poignant as you get when it comes to photos of web developers) was taken as we left work on the pursuit of Ice-cream.

2012-04-09 00:16:35