Hi,
First you need to find why your ranking goes down.
There are lots of factor consider while Google take your website in first page.
Following are some tips for you.
Instead of building more link first find list of link that targeted to your website, and if you found any broken or bad link then remove them immediately.
Check your web content, check your website content is it original or copied, if copied then update unique and user friendly content for your site.
Thanks