Indian Journal of Science and Technology
DOI: 10.17485/ijst/2015/v8i36/79858
Year: 2015, Volume: 8, Issue: 36, Pages: 1-7
Original Article
Md. Abu Kausar1* , V. S. Dhaka1 and Sanjeev Kumar Singh2
1 Department of Computer and System Sciences, Jaipur National University, Jaipur - 302017, Rajasthan, India; [email protected] , [email protected]
2 Department of Mathematics, Galgotias University, Gr. Noida - 201306, Uttar Pradesh, India; [email protected]
Search engines store information locally with the purpose of deliver quick, accessible search abilities. This information is collected by Web crawler. Web crawling is necessary for the maintenance of complete and latest web document gathering for a web search tool. The web document modifies its content on regular basis hence it becomes necessary to build up a successful framework which could identify these sorts of changes proficiently in the most minimal scanning time to accomplish this modifications. The essential thought behind designing of such a web crawler is to find high quality web documents within limited time frame. The proposed system works on the Client-Server Technology it reduces the overlap problem and downloads high quality web pages. We can add many web crawler in parallel to download web page in parallel way.
Keywords: Client-Server Technology, Overlap, Search Engine, Web Crawler, Web Page
Subscribe now for latest articles and news.
