Google I/O 2008 – App Engine Datastore Under the Covers with notes

This discussion is useful for App Engine developers who need to understand the platform.

A sharded, sorted, array.

Sharded: rows are split across machines. Normal relational databases usually shard by row.

Entities table:
Contains all entities for all applications.
Ordered by key so all the related data is grouped together.
Key controls the physical location of an entity in Bigtable. Entities keys exactly map to a physical location in Bigtable.

Indices map property values and optionally ancestors and kinds to entities themselves. Indices include the Entity key. Indices serve query results.

No filtering in memory.

Scans are prefix or range. In looking at searches, look at this seriously.

Data is organized by kind (i.e., the class).

Single-property index serves the exact results in exactly the right order. They are prestored in ascending and descending orders.

Define indicies in app.yaml if not run in the development environment.

I found this very important to understand how to structure my queries and to understand the engine conceptually.

If a query requires an index that does not exist a “Needs Index” error. The error will let the develop know what the index is that is required but non-existent.

Keep entity groups small.
Set properties that are not searched on, use text and blobs, which are not indexed instead of other types.
Everything is done synchronously. The “put” doesn’t return until the db is consistent.


About rico49

Writer, progresive activist, open source software developer. Working to meet the needs of under- and un-employed people globally and in the United States.
This entry was posted in Technology and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s