Shell Script to navigate sub-directories and move directories

Hi guys, I currently have a Ubuntu Server set up that I'm using as a media server. I have movies and music that transfer over to the server in a general "downloads" folder. Then I move the folders over to the respective areas, Music files to the music folder, Videos to the video folder, etc. I get tired of manually moving folders and would like to automate this task a bit. I don't have much experience writing shell scripts, but I started writing a bash script for this task, but alas, it doesn't work! Here is what I have so far:

#!/bin/bash
CWD='pwd'
clear
echo "Starting..."
cd /home/glen/downloads
ls -l
for directory in /home/glen/downloads
do
	for file in directory
		do
			if [ "${file}" == "*.mp3" ] || [ "${file}" == "*.flac"]
			then
				cp directory /home/glen/Music
			else if [ "${file}" == "*.avi" ]||[ "${file}" == "*.mkv" ]||[ "${file}" == "*.mp4" ]
			then
				cp directory /media/sdb/Videos
		fi
echo "Finished!"
cd $CWD
done

If anyone could point me in the right direction I would be very grateful!