You can restrict FTP access by IP through a .ftpaccess file.
# Simple .ftpaccess file to control which IPs
# can access this directory structure
If this method doesn’t work you can also use iptables. However, this is only a valid option if you are implementing this feature server-wide. To block all inbound access via ftp use the following:
iptables -I INPUT -i eth0 --protocol tcp --dport 21 -j DROP;
Then to add an IP for access use:
iptables -I INPUT -i eth0 -s $IP --protocol tcp --dport 21 -j ACCEPT;