Introduction to Google Cloud Machine Learning Engine

TensorFlow

TensorFlow website: https://www.tensorflow.org
TensorFlow installation: https://www.tensorflow.org/install

python -V       # Check which version of Python 2 is installed
python3 -V      # Check which version of Python 3 is installed
pip install --user --upgrade pip
pip install --user --upgrade virtualenv
virtualenv mlenv
source mlenv/bin/activate
pip install tensorflow==1.10
pip install pandas

git clone https://github.com/cloudacademy/mlengine-intro.git
cd mlengine-intro/iris/trainer
python iris.py

Training a Model with ML Engine

Google Cloud SDK installation: https://cloud.google.com/sdk

cd ..
gcloud ai-platform local train --module-name trainer.iris --package-path trainer

BUCKET=gs://[ProjectID]-ml  # Replace [ProjectID] with your Google Cloud Project ID  
REGION=[Region]  # Replace [Region] with a Google Cloud Platform region, such as us-central1  

gcloud ai-platform jobs submit training iris1 \
    --module-name trainer.iris \
    --package-path trainer \
    --staging-bucket $BUCKET \
    --region $REGION \
    --runtime-version 1.10

Feature Engineering

Google's original sample code: https://github.com/GoogleCloudPlatform/cloudml-samples/tree/master/census

cd ../census/estimator

gcloud ai-platform local train \
    --module-name trainer.task \
    --package-path trainer \
    -- \
    --train-files data/adult.data.csv \
    --eval-files data/adult.test.csv \
    --model-type wide

A Wide and Deep Model

gcloud ai-platform local train \
    --module-name trainer.task \
    --package-path trainer \
    -- \
    --train-files data/adult.data.csv \
    --eval-files data/adult.test.csv \
    --model-type deep

Distributed Training on ML Engine

Hyperparameter Tuning: https://cloud.google.com/ml-engine/docs/concepts/hyperparameter-tuning-overview

gsutil cp -r gs://cloudml-public/census/data $BUCKET  
TRAIN_DATA=$BUCKET/data/adult.data.csv  
EVAL_DATA=$BUCKET/data/adult.test.csv  
JOB=census1  

gcloud ai-platform jobs submit training $JOB \
    --job-dir $BUCKET/$JOB \
    --runtime-version 1.10 \
    --module-name trainer.task \
    --package-path trainer \
    --region $REGION \
    --scale-tier STANDARD_1 \
    -- \
    --train-files $TRAIN_DATA \
    --eval-files $EVAL_DATA

Deploying a Model on ML Engine

gcloud ai-platform models create census --regions=$REGION  
gsutil ls -r $BUCKET/census1/export  

# Note: Replace [Path-to-model] below with your Cloud Storage path
gcloud ai-platform versions create v1 \
    --model census \
    --runtime-version 1.10 \
    --origin [Path-to-model]

gcloud ai-platform predict \
    --model census \
    --version v1 \
    --json-instances \
    ../test.json

Conclusion

Cloud Machine Learning Engine documentation: https://cloud.google.com/ai-platform/docs

how to ssh Raspberry Pi with MacOSX

The ssh should be a default package with the RPi. You could check the service by the command.

ps -aux | grep sshed

 If you can't find the service, you could simply install it by the apt-get command.

sudo apt-get install ssh

  And what you have to do is to start the service.

sudo /etc/init.d/ssh start

  Sometime you may want to restart the service.

sudo /etc/init.d/ssh restart

  Then you could ssh from the Macos host side.

  ssh pi@ // you may have to check the ip from the RPi terminal first by the ifconfig command.

  You may want to force the ssh run every time bootup. Run this command once:




sudo update-rc.d ssh defaultssudo reboot

  How to debug if ssh doesn't work.

i) After the RPi is boot up, checking the sshd is running or not. If it is not running, just simply restart the service.

ii) Try to ping the RPi fro the host side.

iii) If the connection is still refused, try to clear the rsa key on the RPi & try again. These are the commands with the root privilege.

sudo -i

root@raspberrypi:/home/pi# ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key

root@raspberrypi:/home/pi# ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key

root@raspberrypi:/home/pi# ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key

service sshd reload.

  If you want to install the ssh from start, there is a good tutorial.

http://youtu.be/SmMMKojOE4U 

Setup arm tool chain for RaspBerry-Pi (with MacOSX host)

Installing the ARM Toolchain

1) Simply download the tool chain package from the link.

https://github.com/downloads/UnhandledException/ARMx/ARMx-2009q3-67.tar.bz2

2) un-tar the package /opt/arm-linux-tools

3) add the tool chain path to the system path env & update the PATH env, e.g.
export PATH=/opt/arm-linux-tools/bin:$PATH

4) This is an option to install the eclipse as a IDE. http://www.eclipse.org/downloads/
Any Eclipse IDE for C / C + + Developers are OK. My version is

Eclipse IDE for C/C++ Developers

Version: Juno Service Release 2

5) After the eclipse, simply go to the HELP menu and checks for the update.

6) Now you could new a C project and set up the cross compile env.

7) Go to the project explorer & select the properties. Set up the cross compiler to "arm-none-linux-gnueabi-" & the cross compiler path.

8) You could add your source codes by New command & then Source File. e.g. main.c

#include

int main(void) {

printf("Hello World!\n");

return 0;

}

9) Then you could compile the program by Build command & verify the compiled codes with command  "arm-none-linux-gnueabi-objdump -t filename" on your host console.

10) You could not run your code in the host as it is a arm executable. 

11) You have to ftp the code to your lovely RPi & you have to setup the ftp server on the RPi but it is not difficult. And then execute the program on the target.

12) You could even setup the remote debugger in the eclipse ^_^

Ubuntu 12.04 stuck at VMware easy install screen!

The issue happened right just after the Ubuntu 12.04 LTS installation, it stuck at VMware Easy install screen.




******************************************************************
******************************************************************
Vmware Easy Install

PLEASE WAIT! VMware Tools is currently being
installed on your system. Depending on the
version of Ubuntu you are installing, you may
log in below and use the system during
intallation. Otherwise, please wait for the
graphical environment to launch. Thank you.

******************************************************************
******************************************************************

  This is the note to fix the issue.

1. Restore the /etc/issue file:
   sudo mv /etc/issue.backup /etc/issue*
2. Restore the /etc/rc.local file:
   sudo mv /etc/rc.local.backup /etc/rc.local
3. Restore the /etc/init/lightdm.conf file:
   sudo mv /opt/vmware-tools-installer/lightdm.conf /etc/init

Then reboot.

Highlight to setup Android build environment with Ubuntu 12.04

  This is a note how to setup the build environment on top of the Ubuntu 12.04.

  I am not going to write down all the details here as most of the steps & details could be found in http://source.android.com/source/initializing.html. Instead I just highlight some common errors here.

i) Install the Java run time engine.

  As the Ubuntu 12.04 is came with the OpenJava run-time & the android build environment is requested a java6 runtime, it will be a problem. Some websites mentioned that user could re-install the jdk with the repo but it just doesn't work. Anyway, the solution is to download it directly from the Java_Archive. You may choose the rpm or bin version. My suggestion is to pick up the bin version as you may need to install the rpm before.

  This is the installation procedures for the jdk.

#change to /opt
#chmod a+x jdk-6u35-linux-x64.bin
#./jdk-6u35-linux-x64.bin

  The above command will unzip the packages. Then what you need is to add the JAVA HOME path to the system. This is a suggestion.

#sudo gedit /etc/profile

  Then add the following lines at the end of the file.

export JAVA_HOME=/opt/jdk1.6.0_35
CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/lib/ft.jar:$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOE/bin:$PATH

  Save the file & reboot the system, it should be fine now.

  To check the jdk version, simply use the following commands.

java -version & u will see the below response.


alexyu@ubuntu:/opt/jdk-6u/jdk1.6.0_35$ java -version
java version "1.6.0_35"
Java(TM) SE Runtime Environment (build 1.6.0_35-b10)
Java HotSpot(TM) 64-Bit Server VM (build 20.10-b01, mixed mode)
alexyu@ubuntu:/opt/jdk-6u/jdk1.6.0_35$

how to write fb?

This is an example showing how to verify the functionality of a embedded system fb. To compile the below code, this is the command.

"$CROSS_COMPILE"gcc fbapp.c -o fbapp

#include ...

#define ERROR(x) printf("fbtest error in line %s:%d: %s\n", __FUNCTION__, __LINE__, strerror(errno));

#define FBCTL(cmd, arg) \
if(ioctl(fd, cmd, arg) == -1) { \
ERROR("ioctl failed"); \
exit(1); }

#define FBCTL0(cmd) \
if(ioctl(fd, cmd) == -1) { \
ERROR("ioctl failed"); \
exit(1); }

struct fb_var_screeninfo var;
struct fb_fix_screeninfo fix;

int open_fb(const char* dev)
{
int fd = -1;
fd = open(dev, O_RDWR);
if(fd == -1)
{
printf("Error opening device %s : %s\n", dev, strerror(errno));
exit(-1);
}

return fd;
}

static void draw_pixel(void *fbmem, int x, int y, unsigned color)
{
if (var.bits_per_pixel == 16) {
unsigned short c;
unsigned r = (color >> 16) & 0xff;
unsigned g = (color >> 8) & 0xff;
unsigned b = (color >> 0) & 0xff;
unsigned short *p;

r = r * 32 / 256;
g = g * 64 / 256;
b = b * 32 / 256;

c = (r << 11) | (g << 5) | (b << 0);

fbmem += fix.line_length * y;

p = fbmem;

p += x;

*p = c;
} else {
unsigned int *p;

fbmem += fix.line_length * y;

p = fbmem;

p += x;

*p = color;
}
}

void fill_screen(void *fbmem)
{
unsigned x, y;
unsigned h = var.yres; //yres_virtual;
unsigned w = var.xres; //xres_virtual;
  // int color;
  for (y = 0; y < h; y++) {
for (x = 0; x < w; x++) {
for (y = 0; y < h; y++) {
for (x = 0; x < w; x++) {
  draw_pixel(fbmem, x, y, (y? 0x00FFFFFF:0));}
        }
}
}
}

int main(int argc, char** argv)
{
int fb_num;
char str[64];
int fd;

if (argc == 2)
fb_num = atoi(argv[1]);
else
fb_num = 0;

sprintf(str, "/dev/graphics/fb%d", fb_num);
printf("opening %s\n", str);

fd = open(str, O_RDWR);

FBCTL(FBIOGET_VSCREENINFO, &var);
FBCTL(FBIOGET_FSCREENINFO, &fix);

printf("res %d,%d virtual %d,%d, line_len %d, bits_per_pixel %d\n",
var.xres, var.yres,
var.xres_virtual, var.yres_virtual,
fix.line_length,
var.bits_per_pixel);

void* ptr = mmap(0, var.yres_virtual * fix.line_length,
PROT_WRITE | PROT_READ,
MAP_SHARED, fd, 0);

if(ptr == MAP_FAILED) {
perror("mmap failed");
exit(1);
}

fill_screen(ptr);
return 0;
}

how-to setup busy box ?

The busybox could be cross compiled for arm-none-linux-gnueabi-gcc.
It is cross compiled with static link option.
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
User could cp the busybox to /system/bin of the target sd card.

//install manually
adb shell
su
mount -o rw,remount -t yaffs2 /dev/block/mtdblock3 /system
exit
adb push busybox /data/local
adb shell
cd /system
ls -al


cd /data/local
chmod busybox 755
/data/local/busybox cp /data/local/busybox /system/xbin/busybox
cd /system/xbin
chmod busybox 755
./busybox --install -s /system/xbin
rm /data/local/busybox