Check partition table for usb drive (i.e. /dev/sdb)
sudo fdisk -l
Delete the partition table and create a new one
sudo fdisk /dev/sdb
Enter o -> w
Create new partitions for OS and encrypted storage
sudo fdisk /dev/sdb
Enter n -> p -> 1 -> 2048 -> +4G -> w sudo fdisk /dev/sdb Enter n -> p -> 2 -> (default) -> (default) -> w
Encrypt the data partition
sudo cryptsetup luksFormat /dev/sdb2
Format the new encrypted partition
sudo cryptsetup luksOpen /dev/sdb2 LUKS0001
sudo mkfs.ext4 /dev/mapper/LUKS0001 -L ENCRYPTED
sudo cryptsetup luksClose LUKS0001
Install your live linux to USB Stick partition 1
Ubuntu Startup Disk Utility can only write one partition, so we use UNetbootin (see links) to install our linux image only to sdb1.