9lessons Programming Blog

Friends just imagine with out Google Search we can't ! Most of Google is implemented in C or C++ for efficiency and can run in either Solaris or Linux.

In Google, the web crawling (downloading of web pages) is done by several distributed crawlers. There is a URLserver that sends lists of URLs to be fetched to the crawlers. The web pages that are fetched are then sent to the storeserver. The storeserver then compresses and stores the web pages into a repository.Every web page has an associated ID number called a docID which is assigned whenever a new URL is parsed out of a web page. The indexing function is performed by the indexer and the sorter. The indexer performs a number of functions. It reads the repository, uncompresses the documents, and parses them. Each document is converted into a set of word occurrences called hits.

High Level Google Architecture

The hits record the word, position in document, an approximation of font size, and capitalization. The indexer distributes these hits into a set of "barrels", creating a partially sorted forward index. The indexer performs another important function. It parses out all the links in every web page and stores important information about them in an anchors file. This file contains enough information to determine where each link points from and to, and the text of the link.

The URLresolver reads the anchors file and converts relative URLs into absolute URLs and in turn into docIDs. It puts the anchor text into the forward index, associated with the docID that the anchor points to. It also generates a database of links which are pairs of docIDs. The links database is used to compute PageRanks for all the documents.

The sorter takes the barrels, which are sorted by docID, and resorts them by wordID to generate the inverted index. This is done in place so that little temporary space is needed for this operation. The sorter also produces a list of wordIDs and offsets into the inverted index. A program called DumpLexicon takes this list together with the lexicon produced by the indexer and generates a new lexicon to be used by the searcher. The searcher is run by a web server and uses the lexicon built by DumpLexicon together with the inverted index and the PageRanks to answer queries.

Next Article LinkGoogle Search Major Data Structures Documentation

What's in a Database?

a database is a container that holds tables and other SQL structures related to those tables.

An easier way to diagram accout table.

Every one having different hobbies. Remove the hobbiles column and put it in its own table

Add name colum that will let us identify which hobbies belong to which person in the account table.

Linking two tables in a diagram

Connecting Two tables

The problem we're trying to use name field to somehow let use connect the two tables. But what if two people in the accout table have the same name ?

Foreing Key Facts :

A foreign key can have a different name that the primary key it comes form.

The primary key used by a foreign key is also knownas a parent key. the table where the primary key is from is knows as a parent table.

The foreing key can be used to make sure that the rows in on table have corresponding rows in another table.

Foreign key values can be null, even thugh primary key values can't.

Foreign key don't have to be unique - in facts, they ofter aren't

CREATE TABLE accout(
user_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(20),
email VARCHAR(30),
birthday DATE,
gender VARCHAR(10),
city VARCHAR(30));

CREATE TABLE hobbies(
hob_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,hobby VARCHAR(100) NOT NULL,
user_id INT NOT NULL,
CONSTRAINT account_user_id_fK
FOREIGN KEY(user_id)
REFERENCES accout(user_id));