robots.txt file

[cire]

---

Guys, do you have an idea where to find the robots.txt file? Can it be found by using filezilla to connect to the server? Or shall I log in to the cpanel? Or to the wp-admin?

I tried to make my own robots.txt file, then when I checked my site url httpsitename)/robots.txt
it is modified as follows
User-Agent: *
Disallow:
but still it is not indexed, it still doesn't allow any bots to crawl the site. And when I tried to delete my own robots.txt file, there still exists a default robots.txt . I don't know where it is generated.

[pauloray]

himo sad ka ka robots nga meta tags:
<meta name="robots" content="INDEX,FOLLOW" />

[cire]

@pauloray
ou naa naman meta tag na robots. sakto na siya. but dili gihapon siya ma index. unsay other reason ani na problem ?

[cire]

In wp-admin, I already allowed the Search Engine Visibility. I also found the functions that generate the robots.txt file, it's in wp-includeds/functions.php . I changed it to allow everything but still, dli gihapon siya ma index.

[pauloray]

tingali wa pa ma-crawl.

Try login sa Google Webmasters Tool. Select your website >> click HEALTH (on the left pane) >> click Fetch as Google >>click FETCH.

itry sad type ni sa Google Search box:

site:www.yourWebsite.com

- if you see results, then na fetch na niya.

Hope that helps.

[cire]

thanks @pauloray! it's working now after submitting some urls to be fetched by Google in the webmasters tools.

[fon&fun]

tingali wa pa ma-crawl.

Try login sa Google Webmasters Tool. Select your website >> click HEALTH (on the left pane) >> click Fetch as Google >>click FETCH.

itry sad type ni sa Google Search box:

site:www.yourWebsite.com

- if you see results, then na fetch na niya.

Hope that helps.

Thanks bro.,,solve already my robot.txt problem.