Sushant's blog

Continuing from the previous part This post continues with details on the role indexes play in SQL queries and how to use indexes effeciently to make our queries better.

As data engineers, our work time is mostly consumed with writing queries that produce the desired output or with designing systems that can properly capture the required data. However, we frequently run across situations where we also need to optimize our queries for efficiency. As entry level data engineers, (or even when we have gained much experience) we might tend to believe that all queries that produce equal output must be equivalent in resource consumption. This is a misconception most of us learn the hard way.

A large benefit of using a Content Management System (CMS) like Drupal is the off-the-shelf modules that can be used to extend the functionality. Drupal has a strong community of developers and has modules for even the most arcane features and if a module doesn’t exist, we either tweak an existing module or build a new one.

Big Talk About BigTable

Relational Databases have hegemony on the way data has been stored. Proponents of Relational Database emphasize on normalization for valid reasons. Maintainability, Integrity (summed up in ACID) and Security has been the primary focus of classical relational database problems. Backed by Moore’s law it was easy to theorize that processing speed would inevitably be a trivial factor over other engineering problems like consistency and integrity. In some sense that is true but hitherto unaccounted for were sites like Google and Facebook which needed to process PetaBytes of data every second in real time.

Applications need to collect data from users. SharePoint emphasizes ease of use for non technical users while providing a comprehensive API for programmers. This means a diversity of techniques that demand different levels of technical complexity and flexibility for design of forms. This article aims to introduce some of these techniques and evaluate them on different aspects like ease of deployment, security etc.

For an urbanite like me who immediately reaches for his phone after hearing a tweet, 27 November 2010 was a welcome change. This day, we at Deerwalk went picnicking near Lakure Bhanjyang. And there, when we heard tweets we looked towards the trees. We were a wild bunch with only one thing on our minds, "to have fun", and we got more than what we had expected.