The final goal of this project is to create a web crawler that can work under the practical environments. In order to achieve that level the project will look in depth to usability and the flexibility of the product.
The final product will be able attract users to the system, and to distribute a client among them. And the web crawler will be capable of collecting information under given key words so that the clients may customize the search patterns according to their needs.
This will provide the common user a web site that he/she might search for web address under a given key word or images under given key words. And for image search the users may select if they want a search according to image tags or page content.
And other developers will be able to download the editable version of the client to edit and distribute a client that is capable of searching a specific area that the developer needs to focus on.
The final deliverables are as followings –
· An online web server application to distribute the workload to the clients.
· An online web site to do the publicity and to distribute the web crawler.
· An Oracle 10G database server to collect the processed information.
· And a Client that will do the crawling and the processing.