Cory O'Daniel - These are just words » sysadmin

Lower ACL Permissions on Amazon S3 items with ruby

Cory O'Daniel — Fri, 12 Feb 2010 01:10:21 +0000

I recently had to change a bunch of permissions on some items on S3. Unfortunately the items were mixed in with items that I didn't want to change the permissions on and the file names are all kinda jumbled, so I couldn't pinpoint what I needed to change by eyeballing the file names.

So I wrote a little ruby method to go in and change something given a key and bucket name.

require 'aws/s3'
 
AWS::S3::Base.establish_connection!(
  :access_key_id     => "YOUR_ACCESS_KEY",
  :secret_access_key => "YOUR_SECRET_KEY"
)
 
include AWS::S3
def s3_set_public_read(key, bucket)
  puts "Processing: #{key}"  
 
  policy = S3Object.acl(key, bucket)
  policy.grants << ACL::Grant.grant(:public_read)
  policy.grants << ACL::Grant.grant(:public_read_acp)
  S3Object.acl(key, bucket, policy)      
end

Now I can just run a loop of ActiveRecord objects around that method and convert them all to public read.

If you want a GUI tool for managing S3 stuff, I totally recommend S3Hub. Its a really sexy tool for managing S3.

Installing MySQL on Mac OSX w/ Ports

Cory O'Daniel — Thu, 11 Feb 2010 19:08:31 +0000

So, I originally installed MySQL on my new MacBook using the DMG as was alluded to in a previous post..

I rebooted my laptop the other day and for some reason all of my tables (InnoDB) where corrupted... I couldn't get them to repair. All the *.frm files were present. Even when I did a 'show tables' they would show up, but whenever I issued any DML mysql responded saying the table did not exist.

So I decided to wipe out my MySQL install, because it just seemed funky. I decided to go the ports route, and I'm so used to the doing the bastardized old way of installing it

1	sudo port install mysql5 +server +devel

Then proceeding to do about 5000 things to get it to start at boot time and have a *.sock file in the right place.

Then I came across this post while looking for a LaunchD file for MySQL. A comment by Mike Richards points out that there is a hip new (working) way of installing mysql with ports:

The mysql5 +server package variant in MacPorts is obsolete, and is superseded by the mysql5-server package, which you install in addition to mysql5. This allows you to build it after the fact, instead of re-compiling the entire mysql5 package with +server.

I'd suggest to just remove your old mysql5 +server, and install using the following.

sudo port install mysql5-server
That'll build both the required mysql5 and mysql5-server packages for you, and you'll have the mysql5 plist file in /Library/LaunchDaemons. Also note that you no longer need to symlink your mysqld.sock to /tmp/mysql.sock.

Thanks Mike!

1	sudo port install mysql5-server

And who would of thunk it? It 'just works' (as a side note, the binary name is mysql5 incase you don't think it works when you type 'mysql').

Make sure you read all the output from Port. You'll need to load the LaunchD file in launchctl and either manually start MySQL the first time or just reboot and have OSX do it for you.

Yay. I can do some work now.

Stick this in your bash

Cory O'Daniel — Tue, 09 Feb 2010 04:23:50 +0000

This isn't a tutorial or anything. Its merely just me dropping all my bash scripts/prompt stuff here so maybe I'll get some comments/tips on cool stuff to add, and so that I can always rip it if I'm on a remote computer.

My pride and joy is that bash prompt. So much info. Its a pretty cool bash prompt if I do say so myself (toot toot)
Username@Host
List of IP Addresses
Number of files in the directory
Current Path
History Command Number
Git Branch (if a git repo)

# ~/.bash_login
# Shell Options
shopt -s checkwinsize
shopt -s cdspell
 
# Exports
export EDITOR="mate -w"
export VISUAL="mate -w"
export HISTFILESIZE=3000
export HISTCONTROL=ignoredups
export DISPLAY=:0.0
export GEM_PATH="/Library/Ruby/Gems/1.8/gems"
 
export PATH=~/bin:/opt/local/bin:/opt/local/sbin:/usr/local/mysql/bin:~/bin:/usr/local/bin:/Applications/flex_sdk_3/bin:$PATH
 
# include functions, aliases & bashrc
if [ -f ~/.functions ]; then
  . ~/.functions;
fi
 
if [ -f ~/.aliases ]; then
  . ~/.aliases;
fi
 
if [ -f ~/.bashrc ]; then
  . ~/.bashrc;
fi
 
if [ -f ~/.bash_prompt ]; then
  . ~/.bash_prompt;
fi
 
if [ -f /opt/local/etc/bash_completion ]; then
  .  /opt/local/etc/bash_completion
fi
complete -C ~/.bash_completion.d/rake -o default rake
 
date
if [ -x /opt/local/bin/fortune ]; then
  /opt/local/bin/fortune -s
fi

# ~/.bash_prompt
# Define a few Color's
BLACK='\[\e[0;30m\]'
BLUE='\[\e[0;34m\]'
GREEN='\[\e[0;32m\]'
CYAN='\[\e[0;36m\]'
RED='\[\e[0;31m\]'
PURPLE='\[\e[0;35m\]'
BROWN='\[\e[0;33m\]'
LIGHTGRAY='\[\e[0;37m\]'
DARKGRAY='\[\e[1;30m\]'
LIGHTBLUE='\[\e[1;34m\]'
LIGHTGREEN='\[\e[1;32m\]'
LIGHTCYAN='\[\e[1;36m\]'
LIGHTRED='\[\e[1;31m\]'
LIGHTPURPLE='\[\e[1;35m\]'
YELLOW='\[\e[1;33m\]'
WHITE='\[\e[0;37m\]'
NC='\[\e[0m\]'              # No Color
 
function drpmpt () {
  first_prompt_line="$WHITE($CYAN\u@\h$WHITE)-($CYAN$(ip)$WHITE)->"
  second_prompt_line="$WHITE($CYAN$(ls -1|wc -l|tr -d "[:blank:]") files$WHITE)-($CYAN\w$WHITE)->"
  third_prompt_line="$WHITE($GREEN!\!$WHITE)$GREEN$(parse_git_branch)$WHITE"
  working_prompt="$first_prompt_line\n$second_prompt_line\n$third_prompt_line> $NC"
 
  PS1=$working_prompt
}
 
PROMPT_COMMAND=drpmpt

# ~/.aliases
# TO BYPASS AN ALIAS DO THE ORIGINAL COMMAND W \, ie \ls
# Aliases
alias pastie='sake pastie:clip'
alias ..='cd ..'
alias igem='sudo gem install --no-rdoc --no-ri'
alias c='clear'
alias dsrm="find . -type f -name .DS_Store -print0 | xargs -0 rm"
alias gcpp='dsrm; git commit .; git pull; git push'
alias sha1sum='openssl sha1'
alias lgem='gem install --no-rdoc --no-ri -i ./gems --ignore-dependencies'
 
alias start_wowza='/Library/WowzaMediaServerPro/bin/startup.sh'
alias stop_wowza='/Library/WowzaMediaServerPro/bin/shutdown.sh'
 
alias hist='history | grep $1'
alias ps='ps aux'
alias home='cd ~'
alias utgz='tar -zxvf'
alias tgz='tar -zcvf'
alias mnts='df -h'
 
# Alias to multiple ls commands
alias la='ls -Al'               # show hidden files
#alias ls='ls -aF ' # add colors and file type extensions
#alias lx='ls -lXB'              # sort by extension
alias lk='ls -lSr'              # sort by size
alias lc='ls -lcr'          # sort by change time
alias lu='ls -lur'          # sort by access time
alias lr='ls -lR'               # recursive ls
alias lt='ls -ltr'              # sort by date
alias lm='ls -al |more'         # pipe through 'more'
 
# Alias chmod commands
alias mx='chmod a+x'
alias 000='chmod 000'
alias 644='chmod 644'
alias 755='chmod 755'

# ~/.functions
 
# Parse git branch
parse_git_branch() {
  git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/ [\1]/'
}
 
# Get assigned ip address
ip(){
OS=`uname`
case $OS in
   Linux) IP=`ifconfig  | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f2 | awk '{ print $1}'`;;
   FreeBSD|OpenBSD|Darwin) IP=`ifconfig  | grep -E 'inet.[0-9]' | grep -v '127.0.0.1' | awk '{ print $2}'` ;;
   SunOS) IP=`ifconfig -a | grep inet | grep -v '127.0.0.1' | awk '{ print $2} '` ;;
   *) IP="Unknown";;
esac
 
# Remove new lines and trailing whitespace.
echo "$IP" | awk '{ printf "%s | ", $0 }' | awk '{ sub(/(\ \|\ )$/, ""); print}'
}
 
#Determine if an app is running
list(){
  ps aux -m -r | grep $1
}
 
 
git_prompt_ip(){
  export GIT_PROMPT_IP=$1
}

Building curb on Snow Leopard

Cory O'Daniel — Wed, 20 Jan 2010 23:32:07 +0000

Had some issues building curb on Snow Leopard today.

It was resulting in something like:

In file included from /opt/local/include/curl/curl.h:44,
                 from curb.h:12,
                 from curb.c:8:
/opt/local/include/curl/curlrules.h:144: error: size of array ‘__curl_rule_01__’ is negative
/opt/local/include/curl/curlrules.h:154: error: size of array ‘__curl_rule_02__’ is negative
lipo: can't open input file: /var/folders/wX/wX64Cb+PGjG-EXuklO+I+k+++TI/-Tmp-//ccKIrqTY.out (No such file or directory)
make: *** [curb.o] Error 1

It's a quick fix and your on your way to curling stuff...

1
2
3

sudo port install zlib +universal;
sudo port upgrade --enforce-variants openssl +universal;
sudo port install curl +universal;

Then build curb however you were building it before.

Yay, I can scrub other peoples sites now!

Installing MySQL from DMG on Mac (and the few commands to make it work on the command line)

Cory O'Daniel — Wed, 20 Jan 2010 22:30:08 +0000

When you install mysql from the DMG on Mac OS X, it leaves a few things missing. Mysql binaries are missing from the path, and the sock file is being looked for in the wrong place by default.

Here is a quick fix:

1
2
3

echo "export PATH=/usr/local/mysql/bin:\$PATH" >> ~/.bash_login;
sudo mkdir -p /opt/local/var/run/mysql5;
sudo ln -s /tmp/mysql.sock /opt/local/var/run/mysql5/mysqld.sock;

Yay, now you can rock out with your bad self.