Introduction to Google Cloud Machine Learning Engine

TensorFlow

TensorFlow website: https://www.tensorflow.org
TensorFlow installation: https://www.tensorflow.org/install

python -V       # Check which version of Python 2 is installed
python3 -V      # Check which version of Python 3 is installed
pip install --user --upgrade pip
pip install --user --upgrade virtualenv
virtualenv mlenv
source mlenv/bin/activate
pip install tensorflow==1.10
pip install pandas

git clone https://github.com/cloudacademy/mlengine-intro.git
cd mlengine-intro/iris/trainer
python iris.py

Training a Model with ML Engine

Google Cloud SDK installation: https://cloud.google.com/sdk

cd ..
gcloud ai-platform local train --module-name trainer.iris --package-path trainer

BUCKET=gs://[ProjectID]-ml  # Replace [ProjectID] with your Google Cloud Project ID  
REGION=[Region]  # Replace [Region] with a Google Cloud Platform region, such as us-central1  

gcloud ai-platform jobs submit training iris1 \
    --module-name trainer.iris \
    --package-path trainer \
    --staging-bucket $BUCKET \
    --region $REGION \
    --runtime-version 1.10

Feature Engineering

Google's original sample code: https://github.com/GoogleCloudPlatform/cloudml-samples/tree/master/census

cd ../census/estimator

gcloud ai-platform local train \
    --module-name trainer.task \
    --package-path trainer \
    -- \
    --train-files data/adult.data.csv \
    --eval-files data/adult.test.csv \
    --model-type wide

A Wide and Deep Model

gcloud ai-platform local train \
    --module-name trainer.task \
    --package-path trainer \
    -- \
    --train-files data/adult.data.csv \
    --eval-files data/adult.test.csv \
    --model-type deep

Distributed Training on ML Engine

Hyperparameter Tuning: https://cloud.google.com/ml-engine/docs/concepts/hyperparameter-tuning-overview

gsutil cp -r gs://cloudml-public/census/data $BUCKET  
TRAIN_DATA=$BUCKET/data/adult.data.csv  
EVAL_DATA=$BUCKET/data/adult.test.csv  
JOB=census1  

gcloud ai-platform jobs submit training $JOB \
    --job-dir $BUCKET/$JOB \
    --runtime-version 1.10 \
    --module-name trainer.task \
    --package-path trainer \
    --region $REGION \
    --scale-tier STANDARD_1 \
    -- \
    --train-files $TRAIN_DATA \
    --eval-files $EVAL_DATA

Deploying a Model on ML Engine

gcloud ai-platform models create census --regions=$REGION  
gsutil ls -r $BUCKET/census1/export  

# Note: Replace [Path-to-model] below with your Cloud Storage path
gcloud ai-platform versions create v1 \
    --model census \
    --runtime-version 1.10 \
    --origin [Path-to-model]

gcloud ai-platform predict \
    --model census \
    --version v1 \
    --json-instances \
    ../test.json

Conclusion

Cloud Machine Learning Engine documentation: https://cloud.google.com/ai-platform/docs

how to ssh Raspberry Pi with MacOSX

The ssh should be a default package with the RPi. You could check the service by the command.

ps -aux | grep sshed

 If you can't find the service, you could simply install it by the apt-get command.

sudo apt-get install ssh

  And what you have to do is to start the service.

sudo /etc/init.d/ssh start

  Sometime you may want to restart the service.

sudo /etc/init.d/ssh restart

  Then you could ssh from the Macos host side.

  ssh pi@ // you may have to check the ip from the RPi terminal first by the ifconfig command.

  You may want to force the ssh run every time bootup. Run this command once:




sudo update-rc.d ssh defaultssudo reboot

  How to debug if ssh doesn't work.

i) After the RPi is boot up, checking the sshd is running or not. If it is not running, just simply restart the service.

ii) Try to ping the RPi fro the host side.

iii) If the connection is still refused, try to clear the rsa key on the RPi & try again. These are the commands with the root privilege.

sudo -i

root@raspberrypi:/home/pi# ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key

root@raspberrypi:/home/pi# ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key

root@raspberrypi:/home/pi# ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key

service sshd reload.

  If you want to install the ssh from start, there is a good tutorial.

http://youtu.be/SmMMKojOE4U 

Setup arm tool chain for RaspBerry-Pi (with MacOSX host)

Installing the ARM Toolchain

1) Simply download the tool chain package from the link.

https://github.com/downloads/UnhandledException/ARMx/ARMx-2009q3-67.tar.bz2

2) un-tar the package /opt/arm-linux-tools

3) add the tool chain path to the system path env & update the PATH env, e.g.
export PATH=/opt/arm-linux-tools/bin:$PATH

4) This is an option to install the eclipse as a IDE. http://www.eclipse.org/downloads/
Any Eclipse IDE for C / C + + Developers are OK. My version is

Eclipse IDE for C/C++ Developers

Version: Juno Service Release 2

5) After the eclipse, simply go to the HELP menu and checks for the update.

6) Now you could new a C project and set up the cross compile env.

7) Go to the project explorer & select the properties. Set up the cross compiler to "arm-none-linux-gnueabi-" & the cross compiler path.

8) You could add your source codes by New command & then Source File. e.g. main.c

#include

int main(void) {

printf("Hello World!\n");

return 0;

}

9) Then you could compile the program by Build command & verify the compiled codes with command  "arm-none-linux-gnueabi-objdump -t filename" on your host console.

10) You could not run your code in the host as it is a arm executable. 

11) You have to ftp the code to your lovely RPi & you have to setup the ftp server on the RPi but it is not difficult. And then execute the program on the target.

12) You could even setup the remote debugger in the eclipse ^_^

Ubuntu 12.04 stuck at VMware easy install screen!

The issue happened right just after the Ubuntu 12.04 LTS installation, it stuck at VMware Easy install screen.




******************************************************************
******************************************************************
Vmware Easy Install

PLEASE WAIT! VMware Tools is currently being
installed on your system. Depending on the
version of Ubuntu you are installing, you may
log in below and use the system during
intallation. Otherwise, please wait for the
graphical environment to launch. Thank you.

******************************************************************
******************************************************************

  This is the note to fix the issue.

1. Restore the /etc/issue file:
   sudo mv /etc/issue.backup /etc/issue*
2. Restore the /etc/rc.local file:
   sudo mv /etc/rc.local.backup /etc/rc.local
3. Restore the /etc/init/lightdm.conf file:
   sudo mv /opt/vmware-tools-installer/lightdm.conf /etc/init

Then reboot.

Highlight to setup Android build environment with Ubuntu 12.04

  This is a note how to setup the build environment on top of the Ubuntu 12.04.

  I am not going to write down all the details here as most of the steps & details could be found in http://source.android.com/source/initializing.html. Instead I just highlight some common errors here.

i) Install the Java run time engine.

  As the Ubuntu 12.04 is came with the OpenJava run-time & the android build environment is requested a java6 runtime, it will be a problem. Some websites mentioned that user could re-install the jdk with the repo but it just doesn't work. Anyway, the solution is to download it directly from the Java_Archive. You may choose the rpm or bin version. My suggestion is to pick up the bin version as you may need to install the rpm before.

  This is the installation procedures for the jdk.

#change to /opt
#chmod a+x jdk-6u35-linux-x64.bin
#./jdk-6u35-linux-x64.bin

  The above command will unzip the packages. Then what you need is to add the JAVA HOME path to the system. This is a suggestion.

#sudo gedit /etc/profile

  Then add the following lines at the end of the file.

export JAVA_HOME=/opt/jdk1.6.0_35
CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/lib/ft.jar:$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOE/bin:$PATH

  Save the file & reboot the system, it should be fine now.

  To check the jdk version, simply use the following commands.

java -version & u will see the below response.


alexyu@ubuntu:/opt/jdk-6u/jdk1.6.0_35$ java -version
java version "1.6.0_35"
Java(TM) SE Runtime Environment (build 1.6.0_35-b10)
Java HotSpot(TM) 64-Bit Server VM (build 20.10-b01, mixed mode)
alexyu@ubuntu:/opt/jdk-6u/jdk1.6.0_35$

how to write fb?

This is an example showing how to verify the functionality of a embedded system fb. To compile the below code, this is the command.

"$CROSS_COMPILE"gcc fbapp.c -o fbapp

#include ...

#define ERROR(x) printf("fbtest error in line %s:%d: %s\n", __FUNCTION__, __LINE__, strerror(errno));

#define FBCTL(cmd, arg) \
if(ioctl(fd, cmd, arg) == -1) { \
ERROR("ioctl failed"); \
exit(1); }

#define FBCTL0(cmd) \
if(ioctl(fd, cmd) == -1) { \
ERROR("ioctl failed"); \
exit(1); }

struct fb_var_screeninfo var;
struct fb_fix_screeninfo fix;

int open_fb(const char* dev)
{
int fd = -1;
fd = open(dev, O_RDWR);
if(fd == -1)
{
printf("Error opening device %s : %s\n", dev, strerror(errno));
exit(-1);
}

return fd;
}

static void draw_pixel(void *fbmem, int x, int y, unsigned color)
{
if (var.bits_per_pixel == 16) {
unsigned short c;
unsigned r = (color >> 16) & 0xff;
unsigned g = (color >> 8) & 0xff;
unsigned b = (color >> 0) & 0xff;
unsigned short *p;

r = r * 32 / 256;
g = g * 64 / 256;
b = b * 32 / 256;

c = (r << 11) | (g << 5) | (b << 0);

fbmem += fix.line_length * y;

p = fbmem;

p += x;

*p = c;
} else {
unsigned int *p;

fbmem += fix.line_length * y;

p = fbmem;

p += x;

*p = color;
}
}

void fill_screen(void *fbmem)
{
unsigned x, y;
unsigned h = var.yres; //yres_virtual;
unsigned w = var.xres; //xres_virtual;
  // int color;
  for (y = 0; y < h; y++) {
for (x = 0; x < w; x++) {
for (y = 0; y < h; y++) {
for (x = 0; x < w; x++) {
  draw_pixel(fbmem, x, y, (y? 0x00FFFFFF:0));}
        }
}
}
}

int main(int argc, char** argv)
{
int fb_num;
char str[64];
int fd;

if (argc == 2)
fb_num = atoi(argv[1]);
else
fb_num = 0;

sprintf(str, "/dev/graphics/fb%d", fb_num);
printf("opening %s\n", str);

fd = open(str, O_RDWR);

FBCTL(FBIOGET_VSCREENINFO, &var);
FBCTL(FBIOGET_FSCREENINFO, &fix);

printf("res %d,%d virtual %d,%d, line_len %d, bits_per_pixel %d\n",
var.xres, var.yres,
var.xres_virtual, var.yres_virtual,
fix.line_length,
var.bits_per_pixel);

void* ptr = mmap(0, var.yres_virtual * fix.line_length,
PROT_WRITE | PROT_READ,
MAP_SHARED, fd, 0);

if(ptr == MAP_FAILED) {
perror("mmap failed");
exit(1);
}

fill_screen(ptr);
return 0;
}

how-to setup busy box ?

The busybox could be cross compiled for arm-none-linux-gnueabi-gcc.
It is cross compiled with static link option.
make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
User could cp the busybox to /system/bin of the target sd card.

//install manually
adb shell
su
mount -o rw,remount -t yaffs2 /dev/block/mtdblock3 /system
exit
adb push busybox /data/local
adb shell
cd /system
ls -al


cd /data/local
chmod busybox 755
/data/local/busybox cp /data/local/busybox /system/xbin/busybox
cd /system/xbin
chmod busybox 755
./busybox --install -s /system/xbin
rm /data/local/busybox

UBIFS how to ^_^

UBIFS howto:


STEP 1:Install UBIFS tools with the follow packages. If you have the mkfs.ubifs then go to

Step 2 Use UBI
1. config kernel option and compile UBI / UBIFS /ext2 into the kernel
(1) Enable UBI and Emulate MTD devices
Path:
"Device Drivers"
-> "Memory Technology Device (MTD) support"
-> "UBI - Unsorted block images",

(2) Enable UBIFS
Path:
"File systems"
-> "Miscellaneous filesystems"
-> "UBIFS file system support"
(2) Enable Ext2
Path:
"File systems"
-> " Second extended fs support "

2. make image
(1)create UBIFS rootfs image
mkfs.ubifs -r filesys/ -m 4096 -e 516096 -c 2047 -o ubifs.img
ubinize -o ubi.img -m 4096 -p 512KiB ubinize.cfg

where ubinize.cfg contains:
[ubifs]
mode=ubi
image=ubifs.img --rootfs image name
vol_id=0
vol_size=128MiB --volume size
vol_type=dynamic
vol_name=ubifs ---UBI volume name
vol_flags=autoresize


3. mount rootfs
(1) mount ubifs root file system on redboot
fis load kernel
exec -c "noinitrd console=ttymxc0,115200 ubi.mtd=2 root=ubi0:ubifs rootfstype=ubifs rw"

(*assume the kernel had 5 partitions. But the UBI will create new partition when attach the UBI ,so root=/dev/mtdblock6)

4 mount a partition via UBI
(1) ubifs
ubiformat /dev/mtdXX
ubiattach /dev/ubi_ctrl -m XX
ubimkvol /dev/ubi0 -N ubi_name -s 512MiB
mount -t ubifs ubi0:ubi_name /mnt/ubifs
ubiupdatevol /dev/ubi0_0 –t - wipe out an existing UBI volume


Note:
About -o sync option in mount, this will make UBIFS to run in synchronous mode, which will disable write-back cache and small write buffers.
If -o sync is not used in mount, then we should run “sync” (symbolic link to BusyBox), to sync write-back cache and small write buffers to NAND, before we reset Dickens.
Above being said, as we don’t have a mechanism to power down gracefully, UBIFS synchronous mode (-o sync) is recommended at this moment.

how to build mtd-utilities

1) Download mtd-utils from the link.

ftp://ftp.infradead.org/pub/mtd-utils/

(PS: Reference link http://elinux.org/CompilingMTDUtils)

2) Preparation for x86 build.

i) zlib
ii) lzo
iii) uuid

(PS: For ubuntu 10.04, user can install the packages from "Synaptic Package Manager" or "apt-get".)

3) Build with the following command.

WITHOUT_XATTR=1 make
sudo su
DESTDIR=/usr/local make install

To compile external mtd-utils-1.3.x, build e2fsprogs, lzo, zlib in ltib first. No need to modify the Makefile.

Then run the following:

Setting the following flags.

ZLIBCPPFLAGS="-I/home/usr/include" ZLIBLDFLAGS="-L/home/usr/lib" CROSS=arm-none-linux-gnueabi- WITHOUT_XATTR=1 make

Adding new U-Boot commands

U-Boot providing interactive commands to end user but it may be necessary to add extra commands. There is a standard command interface
and it is defined in include/command.h. Every command defined a cmd_tbl_t structure.

#define U_BOOT_CMD(name,maxargs,rep,cmd,usage,help) \
cmd_tbl_t __u_boot_cmd_##name Struct_Section = {#name, maxargs, rep, cmd, usage}

struct cmd_tbl_s {
char *name; /* Command Name */
int maxargs; /* maximum number of arguments */
int repeatable; /* autorepeat allowed? */
/* Implementation function */
int (*cmd)(struct cmd_tbl_s *, int, int, char *[]);
char *usage; /* Usage message (short) */
#ifdef CONFIG_SYS_LONGHELP
char *help; /* Help message (long) */
#endif
#ifdef CONFIG_AUTO_COMPLETE
/* do auto completion on the arguments */
int (*complete)(int argc, char *argv[], char last_char, int maxv, char *cmdv[]);
#endif
};

The command structure defined the name of the command, max number of the arguments, autorepeat, implementation function, usage and help.

The command console is defined in common/command.c. The find_cmd() is used to match the correspnding command function.

Below is an example to show how to read the cpu id.

#define read_cpuid(reg) \
({ \
unsigned int __val; \
asm("mrc p15, 0, %0, c0, c0, " __stringify(reg) \
: "=r" (__val) \
: \
: "cc"); \
__val; \
})

int do_cpuid(cmd_tbl_t * cmdtp, int flag, int argc, char *argv[])
{
printf("cpu id: %x\n", read_cpuid(0));
printf("cache type: %x\n", read_cpuid(1));
printf("chip id: %x\n", *(volatile unsigned int *)0xd4282c00);
return 0;
}

U_BOOT_CMD(
cpuid, 6, 1, do_cpuid,
"cpuid - read cpu id\n",
" - read cpu id\n"
);

Uboot for PXA (2)

Compilation for UBOOT
=====================
Pre-request: User should install the corresponding cross tool chain in advance.

It is a very simple procedures, user has to choose the target platform first.

user:/home/user/u-boot-arm/make ARCH=arm CROSS_COMPILE=arm-marvel-linux-gnueabi- aspenite_config

Then user should able to build the uboot as below.

user:/home/user/u-boot-arm/make ARCH=arm CROSS_COMPILE=arm-marvel-linux-gnueabi-

(PS:The output binary is placed in the same directory.)

CPU directory
=============

uboot is starting from \u-boot-arm\cpu\pxa\start.s & it defined the standard arm reset vector table
as below.

.globl _start
_start: b reset
ldr pc, _undefined_instruction
ldr pc, _software_interrupt
ldr pc, _prefetch_abort
ldr pc, _data_abort
ldr pc, _not_used
ldr pc, _irq
ldr pc, _fiq

_undefined_instruction: .word undefined_instruction
_software_interrupt: .word software_interrupt
_prefetch_abort: .word prefetch_abort
_data_abort: .word data_abort
_not_used: .word not_used
_irq: .word irq
_fiq: .word fiq

.balignl 16,0xdeadbeef

Below codes are impoerant & defined control by u-boot-arm\board\pxa\aspenite\u-boot.lds

_TEXT_BASE:
.word TEXT_BASE

.globl _armboot_start
_armboot_start:
.word _start

/*
* These are defined in the board-specific linker script.
*/
.globl _bss_start
_bss_start:
.word __bss_start

.globl _bss_end
_bss_end:
.word _end

Then it is the actual reset codes.

reset:
mrs r0,cpsr /* set the CPU to SVC32 mode */
bic r0,r0,#0x1f /* (superviser mode, M=10011) */
orr r0,r0,#0x13
msr cpsr,r0

Then it will call cpu_init_crit & performing very low level init stuffs. Normally, user no need to touch
these pieces of codes. It init clock, timer, interrupts & then determmine copy itself from flash into RAM (relocation) for execution.

The stack is setup as below.

_TEXT_BASE:
.word TEXT_BASE

.globl _armboot_start
_armboot_start:
.word _start

/*
* These are defined in the board-specific linker script.
*/
.globl _bss_start
_bss_start:
.word __bss_start

.globl _bss_end
_bss_end:
.word _end

Finally, it will jump to the C codes (Congrtulation ^_^)

ldr pc, _start_armboot

The start_armboot function is defined in lib_arm/board.c

cpu.c

It is defined those I-D caches function & doing IRQ setup if necessary.

interrupt.c

The pxa u-boot doesn't setup any interrupts.

serial.c

It is the serial port init, pxa_setbrg_dev().

Board directory
===============

Board directory mainly includes all the board specify stuffs.

u-boot.lds //linker script
aspenite_pxa168.c // development board init.

board_init() in the aspenite_pxa168.c is important, it enables clock, init gpio, regulator ...etc.

lib_arm directory
=================

start.s will finally call start_armboot and it is defined in /lib_arm/board.c

It is mainly call the mem_malloc_init function and all the peripherials init functions.

Conclusion
==========

The uboot is very powerful & it has many features to be explored.

Uboot for PXA (1)

UBOOT is a loader based on PPC boot loader.

The file structure is simply shown as below:

u-boot-arm
|_board
|_common
|_cpu
|_disk
|_doc
|_drivers
|_examples
|_fs
|_include
|_lib_arm
|_lib_avr32
|_lib_blackfin
|_lib_generic
|_lib_i386
|_lib_m68k
|_lib_microblaze
|_lib_mips
|_lib_nios
|_lib_nios2
|_lib_ppc
|_lib_sh
|_lib_sparc
|_libfdt
|_nand_spl
|_net
|_nenand_ipl
|_post
|_tools
...
...
...

board: All supporting target boards ard defined in this directory. For example \u-boot-arm\board\pxa\aspenite
A board_init function is defined in aspenite_168.c

cpu: All supoorting CPU type are defined in this directory. For example \u-boot-arm\cpu\pxa
All CPU env init including interrupts are defined. start.s is the first pieces of codes running
in the system. Its main task is to copy the whole uboot from flash into ram & start execution.

common: This is the directory stored all the uboot commands. We normally look at cmb_boot.c & cmd_bootm.c
It has the function to verify the kernel & doing decompression.)

drivers: This directory provides all the interface drivers.

fs: This directory provides all the supporting file system.

lib_arm: This directory providing all the common libraries for arm platform.

include: All headers are placed in this directory. For example u-boot-arm\include\asm-arm\arch-pxa & u-boot-arm\include\asm-arm\arch-pxa168

A report for 2009 Java Certification Day

The event mainly introduce the updates of the Java technology. There are some interesting stuffs I would like to share with you ^_^

JavaFX:
The JavaFX technologies is great & it is a real cross platform environment. Developers can focus on the applications and no need to concern on the porting issues. JavaFX applications developed on PC could be moved to Mobile phone, MID or even TV very easily. More information could be found in http://www.javafx.com/

http://kenai.com/
It is a service where you can host your open source projects and code, as well as find and collaborate with developers of like mind. The presenter just demostrat a few steps to clone & creatre a facebook application & it is real interesting. As you don't need to understand PHP or some other coding language — such as Ruby on Rails, JavaScript, or Python — especially one that has a client library for our API.

http://www.java.com/en/store/index.jsp
In future, Java developers can sell the applications just similar to the iPhone developer. ^_^ It is another platform to support freelance developer.

http://zembly.com/
It is a super great service. It inspire the developer to collaborate with other developers of like mind. It is an excellent collaboration of the software development flow. NetBean 6.7 IDE will support this service. ^_^

LeJOS:
LeJOS (Lego Java OS) is an open source project. It allow you to program LEGO MINDSTORMS NXT in Java. For information could be found in http://lejos.sourceforge.net/index.php

What next?
SUN & Oracle is going to be merge & some new big things are coming. Let see what happen ^_^

First trial to compile Android (Part 2)

1) To install the cross compiler for your target device.
e.g. arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2

It can be downloaded from codesourcery.com

2) Download the Linux kernel linux-2.6.25-android-1.0_r1.tar
$ tar -xf linux-2.6.25-android-1.0_r1.tar

3) Downlaod the Linux version Android SDK android-sdk-linux_x86-1.5_r2

4) To assemble the root file system.
i) To download the busybox under linux environment. http://benno.id.au/blog/2007/11/14/android-busybox
ii) To run the emulator from the SDK & acquire the root file system.
$./tools/emulator &
iii) The Android should be launched & user should go to the Settings & enable the folloing options.
Wait for debugger.
Show running processes.
Show screen updates
iv) Under the console, user should install the busybox as following.
$./adb push ./busybox /data
$./adb shell
$/data/busybox tar -czf system.tar.gz /system
$/data/busybox tar -czf data.tar.gz /data
$/data/busybox tar -czf etc.tar.gz /etc
$/data/busybox tar -czf sbin.tar.gz /sbin
$exit
$./adb pull /system.tar.gz ./
$./adb pull /data.tar.gz ./
$./adb pull /etc.tar.gz ./
$./adb pull /sbin.tar.gz ./
$./adb pull /init ./

5) To create the rootdisk
$ mkdir ram_disk
$ cd ram_disk
$ mkdir cache dev proc root sys tmp var
$tar -xf ./system.tar.gz
$tar -xf ./data.tar.gz
$tar -xf ./etc.tar.gz
$tar -xf ./sbin.tar.gz
$cp ./init ./

The root disk is ready now ^_^

7) The android kenel may not support the target board, user has to change the build configuration & add the platform depend codes accordingly.
e.g. /arch/arm/mach-s3c6410

8) Debug & cross compile the kernel & the rootdisk accordingly.

9) Good luck & have fun ^_^

First trial to compile Android (Part 1)

1) To install the ubuntu-9.04-desktop-i386 system.

2) To install some building packages & utilities.

$ sudo apt-get install flex bison gperf libsdl-dev libesd0-dev libwxgtk2.6-dev build-essential zip curl
$ sudo apt-get install valgrind
$ sudo apt-get install sun-java6-jdk

4) To add some system variables.

export LANG=c
export PATH=~/bin:$PATH:.
export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.07
export ANDROID_JAVA_HOME=$JAVA_HOME


3) It must install the Git & Repo, please read the link in detail. http://source.android.com/download/using-repo

$ sudo apt-get install git-core gnupg
$ curl http://android.git.kernel.org/repo >~/bin/repo
$ chmod a+x ~/bin/repo
$ mkdir myAndroid
$ cd myAndroid
$ repo init -u git://android.git.kernel.org/platform/manifest.git

4) User can download the Android codes now.
$ repo sync



5) To compile the Android.

$make

6) Checking the result in myAndroid/out.

Have fun ^_^

Ubuntu 8.04 Kernel Compilation

1) To prepare the necessary tooling, kernel-package libncurese-dev
>sudo apt-get install kernel-package libncurses5-dev

2) To download kernel source from kernel.org.
>cd /usr/src
>sudo wget http://ww.kernel.org/pub/linux/lernel/v2.6/linux-2.6.24.7.tar.bz2

3) To de-compress the kernel sources.
>sudo tar jxvf linux-2.6.24.7.tar.bz2

4) To create a soft link to the kernel source.
>sudo ln -s linux-2.6.24.7 linux

5) To copy the kernel configuration.
> cd linux
>sudo cp /boot/config-'uname -r' .config

6) To config the kernel.
>sudo make menuconfig

7) To clean up the old stuffs first.
> sudo make distclean

8) Exciting now, start to build the kernel source.
> sudo make-kpkg --initrd kernel_image kernel_headers

9) To install the kernel.
>sudo dpkg -i linux-image-2.6.24.7_2.6.24.7-10.00.Custom_i386.deb linux-headers-2.6.24.7_2.6.24.7-10.00.Custom_i386.deb

10) Aftter installation, user can check /boot/grub/menu.lst to ensure the kernel was added to the menu.

11) Reboot. ^_^

Setup a VMImage based on Ubuntu 8.04 for ARM Linux development.

Below is my blog to setup a VMImage based on Ubuntu 8.04 for accio development.

1) Install the VMWare workstation version 6.x or above. Please use the Bridge for network setting.

2) Since I am using static IP instead of the DHCP, please change the network setting of /etc/network/interfaces accordingly.

auto lo
iface lo inet loopback

#The primary network interface
auto eth0
#iface eth0 inet dhcp

iface eth0 inet static
address "your IP"
netmask 255.255.255.0
network ""
broadcast ""
gateway ""


3) To install the ftp server.

sudo apt-get install proftpd

(PS: Please select to run proftp server from standalone.)


(reference link: http://www.ubuntugeek.com/settingup-an-ftp-server-on-ubuntu-with-proftpd.html)

4) To install the samba server.

sudo apt-get install samba (reference link: http://ubuntuforums.org/showthread.php?t=202605)

5) To install gcc.

sudo apt-get install build-essential (reference link: http://www.linuxforums.org/forum/ubuntu-help/104156-gcc-installation-ubuntu.html)

if internet can't connect, please try.

sudo apt-cdrom add
sudo apt-get update
sudo apt-get install build-essential

6) To install ncurses. (It is for kernel menuconfig.)

sudo apt-get install build-essentials
sudo apt-get install libncurses5-dev

(reference link: http://ubuntuforums.org/showthread.php?t=296224)

7) To install gcc compiler version 3.4.6 for ARM9 Linux Platform
i) Download the arm-linux-gcc-3.4.6-ssl-0.1.2.tar.bz2 to /tmp directory
ii) Unzip the arm-linux-gcc-3.4.6-ssl-0.1.2.tar.bz2
- cd /tmp
- tar jvxf /tmp/ arm-linux-gcc-3.4.6-ssl-0.1.2.tar.bz2 –C /
iii) Set the environment variable of the user.
- Edit ~/.bashrc
- Add the line
ARM_GCC_TOOLS=/usr/local/arm/3.4.6/bin
PATH=$ARM_GCC_TOOLS:$PATH

8) TO install CVS client.

sudo apt-get install cvs

(reference link: http://sanatio.blogspot.com/2005/12/cvs-server-on-ubuntu.html)

(PS: User manual for ubuntu could be found here http://ubuntuguide.org/wiki/Ubuntu:Hardy)

WinCE 5 vs WinCE 6 ^_^

This is reference to http://www.e-consystems.com/WindowsCE5vs6.asp

The major changes in 6.0 are

• Process address space is increased from 32MB to 1 GB.
• Number of processes has been increased to 32K from 32.
• User mode and kernel mode device drivers are possible.
• Device.exe, filesys.exe, GWES.exe has been moved to Kernel mode.
• SetKMode and set process permissions not possible.
• System call performance will improve.

And here is the details:

- It supports safe SEH for security compliance (/GS)
- Secure C Run-Time Libraries
- PREfast support for ARM, MIPS, and SH
- Increase from 32 to 32,000 processes on the device
- Instead of a single 2GB VM for all processes now 2GB per process
- Separation of user and kernel mode space
- File system can support data encryption
- Added 802.11i support for WPA2 compliance
- Windows Media DRM 10 PD and ND
- Graphic windowing and events subsystem (GWES)
- Networking (TCP/IP, IPv6)
- Expanded file system
- More device drivers
- Critical OS components moved into kernel space (critical drivers, file system, and graphical window manager into the kernel)
- New shared heap (R/W for kernel R/O for user land)
- 1GB per process space
- New operating system layout
- In-depth security review of: System calls, Handles, Exception Handling, Memory Allocation, Loader,
- Improved parameter validation for system calls
- Per-Process page and handle tables - greatly improves process isolation, Improves code robustness
- System calls run on special kernel side stacks
- Safeguards system calls from stack tampering
- Heap control structures separated from heap data
- Safe Remote Heaps for OS components
- OS servers can open heaps in user process
- R/W for servers, R/only for user
- Performance optimization and safety from tampering
- Support for secure boot loaders
- Public / private key based
- Based around file signing
- Features supported in hardware (partial list)
- On-chip, tamper-resistant boot ROM
- On-chip RAM (secure key storage)
- Cryptographic support
- JTAG can be disabled
- Fusible or one-time programmable elements
- They acknowledge if JTAG is enabled, it all "falls apart"
- There are more complex attack scenarios that require secure PCB design to mitigate
- Embedded (nested) pointers and how they can be used for attack, such as:
- Pass an embedded pointer to some kernel address space and ask the driver to read/write to the addressed buffer, thereby potentially modifying the kernel
- There is a lot of emphasis around validation to mitigate this
- The kernel will perform parameter pointer validation
- It’s down to the API to perform embedded pointer validation
- The thread permissions model has changed in drivers
- There is strong emphasis on copying the "callers buffer" and then using that copy, as opposed to using the callers buffer itself

開場白

For a opening speech, I think about it for quite a while. However, I still can't figure out what to say.

Anyway, I wanna to share my programming experience here with anyone whose do have interest in embedded system programming.

Good Luck.