Skip to main content

Hi All, 

 

I am trying to use the 

Web Content (Crawler) connection to crawl over a static web page to read the data. Also I have established the connection by using the Basic Authentication

 method. 

 

And then I am creating the Data Stream, DLO and DMO. But when I am refreshing the Data Stream, I am not getting the actual data from the page. 

 

The Data I am getting from the page as per below: 

 

1. When I am 

not using

 "Included Page Element", "Included Web Pages" in the Data Stream, then I am getting the some other data that is not present in that page. 

2. When I am 

using any of them or both

("Included Page Element", "Included Web Pages") in the Data Stream, I am not getting any data. 

 

N.B : 

a) I have tried by setting the crawl depth to 0,1 and 2.

b) In the connection, I have tried by checking and unchecking the "Ignore robots.txt" options. 

 

The website which I am trying to crawl, contains the data in HTML format. But still I am not getting the exact data in the DLO. 

 

If anybody can provide any possible solutions or reference, that would be helpful. 

 

Thanks in advance!!!   

 

@* Data 360 * 

2 answers
0/9000