Hi All,
I am trying to use the
Web Content (Crawler) connection to crawl over a static web page to read the data. Also I have established the connection by using the Basic Authentication
method.
And then I am creating the Data Stream, DLO and DMO. But when I am refreshing the Data Stream, I am not getting the actual data from the page.
The Data I am getting from the page as per below:
1. When I am
not using
"Included Page Element", "Included Web Pages" in the Data Stream, then I am getting the some other data that is not present in that page.
2. When I am
using any of them or both
("Included Page Element", "Included Web Pages") in the Data Stream, I am not getting any data.
N.B :
a) I have tried by setting the crawl depth to 0,1 and 2.
b) In the connection, I have tried by checking and unchecking the "Ignore robots.txt" options.
The website which I am trying to crawl, contains the data in HTML format. But still I am not getting the exact data in the DLO.
If anybody can provide any possible solutions or reference, that would be helpful.
Thanks in advance!!!
,
Thanks for reaching out. My issue got resolved.