Avoid git prompt for username and password

While working with git you can either enter username and password at each time you perform operations like git pull, push, etc.  In the following example, I will be using github repository for remote  with local repos on my Mac.  I will also be using my own account and company user account to access two different github accounts and their respective repos.

Simple setup:

   One account only and access to its respective repositories.

Here just modify your ~/local_repo/.git/config file to either use https or ssh connection.  Typical config looks like
http://www.google.com/jsapi console.log(“JS Test again”); /***** function httpGet(theUrl) { var xmlHttp = null; xmlHttp = new XMLHttpRequest(); xmlHttp.open( “GET”, theUrl, false ); xmlHttp.send( null ); return xmlHttp.responseText; } // google.load(‘visualization’, ‘1’); google.load(‘visualization’, ‘1’, {‘packages’:[‘motionchart’]}); // google.setOnLoadCallback(drawTable); var gvizOpt = {}; gvizOpt[‘width’] = 200; gvizOpt[‘height’] = 100; // gvizOpt[‘showSidePanel’] = true; // gvizOpt[‘showXMetricPicker’] = true; // gvizOpt[‘showYMetricPicker’] = true; // gvizOpt[‘showXScalePicker’] = false; // gvizOpt[‘showYScalePicker’] = false; // gvizOpt[‘showAdvancedPanel’] = false; gvizOpt[‘state’] = ‘{“iconKeySettings”:[], “iconType”:”BUBBLE”, “colorOption”:”_UNIQUE_COLOR”, “showTrails”:false}’; function drawTable() { // Construct query var query = “SELECT ‘date’ as Date, ” + “‘schema_name’ as schema_name, ‘t_size’ as Number” + ‘FROM 13u7ahigSN6LR3nuakbDIQCt81iMZIw3X5IvIvyU’; // ‘FROM test-1mFAkahVNLdAfVEUZ2-AgCyJZZqU_721ELV6NRwY’; var queryText = encodeURIComponent(query); var str_data = httpGet(‘https://www.googleapis.com/fusiontables/v1/query?key=AIzaSyCERZgKBKIohmvrtFcwTvSxrShqA2vmF8U&sql=SELECT%20schema_name,%20date,%20t_size,%20d_size%20FROM%2013u7ahigSN6LR3nuakbDIQCt81iMZIw3X5IvIvyU’); var data = JSON.parse(str_data); console.log(data); // var gvizQuery = new google.visualization.DataTable(); // gvizQuery.addColumn(‘string’, data.columns[0]); // gvizQuery.addColumn(‘date’, data.columns[1]); // gvizQuery.addColumn(‘number’, data.columns[3]); // gvizQuery.addColumn(‘number’, data.columns[2]); // data.rows.forEach(function (ele, ind, arr) { ele[1] = new Date(ele[1]); ele[2] = parseInt(ele[2]); ele[3] = parseInt(ele[3]); gvizQuery.addRow(ele); }); var motionchart = new google.visualization.MotionChart(document.getElementById(‘chart_div’)); // // motionchart.draw(gvizQuery, {‘width’: 900, ‘height’: 600}); // motionchart.draw(gvizQuery, gvizOpt); } google.setOnLoadCallback(drawTable); console.log(“JS after httpget call”); *****/

DW Schema Sizes
[core]
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
    ignorecase = true
[remote "origin"]
    fetch = +refs/heads/*:refs/remotes/origin/*
    # url = https://github.com/USER_OR_ORG_NAME/REPO.git
    url = ssh://git@github.com/USER_OR_ORG_NAME/REPO.git
[branch "master"]
    remote = origin
    merge = refs/heads/master
...
...

With command line git pull/push and ssh pub-key setup at github.com.  See how to provide the pub-key to github.  If you already have a ssh key and you would like to use the same just add entry in the ~/.ssh/config file with corresponding entries.  See below for more details on how to do this.

By default when cloning is done with git clone command the config file generated will have entry of “url = https://github.com”  (assuming you are cloning from github repo).   See how to switch from https to ssh for a given repo or you can comment out the line as above and enter “ssh” link shown above in the example.  Git config (git_config) has too many options and you can control different aspects of git behavior.

Multiple account setup:

Among many but another scenario is when using two or more different github accounts from the same computer/system.  Under this ~/.ssh/config file will be useful.  Here ssh setup is required and https doesn’t help much.

Other variations would be multiple accounts with different service providers not just github.  This is little bit easier than having multiple accounts with same service providers.

Example: Github two accounts – doe_work and doe_personal and for each account you create two separate ssh keys with ssh-keygen.

> ssh-keygen -t rsa -f ~/.ssh/id_rsa_doe_work_git  -C “Job new_key doe@company.com”
> ssh-keygen -t rsa -f ~/.ssh/id_rsa_doe_personal   -C “Personal key”

Now configure both accounts to use ssh and its config – ssh_config.  Important section in ssh_config is the “host” which is a string pattern to match in the config file to get it’s respective options/ variables.  You obviously should add the newly created public keys to github under settings->SSH Keys->Add SSH Key. Sample pub key for above.

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdnbxKkCrYUv3YbutC2Dw6jIhQWLNIzNA3Ec6inlmrngwB33fCaEP4ZiOzPq8A0BRBCyV HYhC3txA9Jn1tRXVZ4tUGEslvN2qF2HNXJhSx8V5Vk1r3LmWe1uehOjAekSK0apELpkafSwigzgkm9oAmbNQ5p0N1e8ar/TXbOOzWVMRu9K G/fILuHf90UZ4H5hOrZov9eZSwabnSMvORirizFXYZPp/FQ30fV3wZJKJoNnmOY+/txjnNc+mikYiezjeA66vWlDGfJQ+Xlb+i1bnXoxBfv hrE/nSuSUVNmGy0bYPOFwbxPrnz0jFGCgdUh7KfKD2yE/gc0abhW0nyxkP Job new_key doe@company.com

...
ServerAliveInterval 60
ServerAliveCountMax 60
...
Host github.com
    HostName github.com
    # User doe_work
    IdentityFile ~/.ssh/id_rsa_doe_work_git
...
Host github-mine
    HostName github.com
    # User doe_personal
    IdentityFile ~/.ssh/id_rsa_doe_personal
...
Now in each repo change /.git/config to use the above ssh/hosts.
[remote "origin"]
    fetch = +refs/heads/*:refs/remotes/origin/*
    url = ssh://git@github.com/USER_OR_ORG_NAME/REPO.git
and
[remote "origin"]
    fetch = +refs/heads/*:refs/remotes/origin/*
    url = ssh://git@github-mine/USER_OR_ORG_NAME/REPO.git

The three configuration files discussed are used by git to perform handshake with remote server and authenticate for each of git pull, push, fetch etc.

Users Password Analysis

As a data engineer it is always interesting to work on large unique data sets.  With recently released Yahoo users details (453K) many insightful info can be gleaned from the data.  For example, even though password hacking is well known for long time still large number of users use simple passwords, sometime as simple as “password” or “123456” or similar.  Here are top 10 passwords and number of users who had used them!

123456         1667
password       780
welcome        437
ninja              333
abc123          250
123456789   222
12345678     208
sunshine        205
princess         202
qwerty          172 

It is interesting to see how many users had unique passwords which was not used by anyone in this data set.  There were 10.6K users with no password which might be due to data issue and ignored for many of calculations and only ~304K (69%) users with unique passwords.

Another interesting insight is if password is used by more than one user, there is likely hood that it is some kind of latin word or words (“whatever”, “iloveyou”) or proper name (“jordon”, “ginger”) or some number (123321) or what can easily be guessed (for example, “q1w2e3r4” for qwerty keyboard or “asdfgh”, etc.).  Even when two users used the same password there was some certainty that it is a guessable password! With each additional user the certainty increases quite quickly.  Under these circumstances, even if a password is encrypted (by md5 or sha or other encryptions) by service providers, with brute force application one can find out the password for these users.

By also looking into how users from different email service providers had their passwords setup showed the following.  As expected, Yahoo had more users (x-axis) while smaller companies (“others” in the chart) had more number of users (71.7%) with unique passwords.  At the same time gmail and live users’ password length is more than 8.87.  Length of the passwords is represented by size of the bubble.

Having bigger bubble size and higher up in the Y-axis is better as it represents more users using unique passwords with longer password strings.  See table below for more details.

Even more interesting analysis can be done including people’s or places’ names in their password.  One could be able to use popular names from US Social Security Administration’s and names’ list go as back as 1880! There were lot more passwords that simply used these names!  Lot more matches can be found with minor modifications like changing i to 1 or o to 0 (zero), etc.

With many users using simple passwords service providers or websites should force each user to have stronger password by enforcing them during the registration or each login.  Users should also be forced change them once in few months.  It might be even better each computer equipped with finger or eye reader that can be used for user authentication thus avoiding this whole password mess.

MySQL Connection over SSH

When MySQL server is setup to accept only localhost connection through bind_address in /etc/mysql/mysql.cnf (configuration)

# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address          = 127.0.0.1

or you want to connect as “root” to mysql and use GUI clients like Workbench or Toad, etc. you can use SSH tunneling.

In operating systems that provide built-in ssh support it is made little easier.  Mac OS is one such OS, which I will use as an example below.

Assuming you have already setup ssh connection, run the following command from the local terminal. If you have not setup the ssh, see this brief introductory ssh article. Or run > man ssh for more details.

prompt>  ssh -L 3307:localhost:3306  user@remote.com

and then let this terminal running.  Here I am using ssh -L [bind_address:]port:host:hostport  parameter and skipped bind_address which is needed only when you have more than one IP attached (multi-IP) to your local system which is typically not the case.

    3307 – The port that you will use on the other clients that need to connect to remote mysql. For example, Workbench.

    localhost:3306 – The remote login details as if you had connected to it through ssh and connecting to mysql instance running on port 3306.  If you had successfully, ssh-ed to remote host at the prompt you would have run > mysql -h localhost -u user_name -P 3306

    user@remote.com – Remote user name and remote host

Now start Workbench and setup new connection string by providing the details as if you are connecting to a mysql instance on your local machine running on port 3307. See figure 1.  ssh will automatically tunnels your connection to remote host’s mysql! See figure 2.  On ending the tunneling setup either my closing the terminal where you were running “ssh -L” command and killing it will disconnect the connection to Workbench.

Figure 1.

Figure 2.

Note: Same technique can be used to for other client applications to connect to any of remote applications/databases over SSH.

Have fun,

Shiva