Hi from Dwipal

Understanding PAUSD Budget Dynamics

Tue, 03 Mar 2026 10:00:00 -0800

PAUSD Has $35K Per Student — A Number Most Districts Would Kill For. Why Doesn’t It Feel Like It?

Most school districts in America would do backflips for $35,000 per student. Palo Alto gets that — and then some. In comparison, Los Altos spends about $28K, San Jose around $18K, and the national average is about $18K. A decade ago, PAUSD was at roughly $18K — about $23K adjusted for inflation.

Yet every budget cycle brings the same story: projected shortfalls, threats of teacher layoffs, and heated board meetings… and calls for yet another parcel tax increase.

How can a district spending nearly double the national average still feel chronically broke? I looked at the data, and also built PAUSD Budget Explorer to understand where the money goes.

From the latest budget (2024–25):

Total operating budget: $340 million
Enrollment: ~10,200 students
Per-student spending: up from ~$18K just a decade ago
National average: ~$18K (Statista/NCES)

At the same time:

Enrollment has dropped from 12,261 to 10,209 students
Staff full-time equivalents have risen from 1,300 to 1,384

Why the Extra Money Per Student Isn’t Translating to Better Schools?

PAUSD doesn’t get paid by student count, it gets paid by property taxes. So when enrollment falls, there is more money available per student. Hypothetically, this should be a gift: smaller classes, more support staff per child, richer programs.

I pulled NCES federal data for PAUSD from 2014 and 2024. Here’s what a decade looks like:

Classroom teachers dropped from 681 to 597 — a loss of 84 teaching positions, even as per-student funding nearly doubled. Meanwhile, total staff grew from roughly 1,256 to 1,385 — an increase of about 128 positions. That means the district cut 84 teachers and added over 200 non-teaching positions.

Where did the growth go?

Instructional aides: 194 → 247 (+53)
School administrators: 27 → 35 (+8)
School admin support staff: 51 → 71 (+20)
Other support services: 175 → 293 (+118)
School psychologists: not separately tracked in 2014, now 19.4 FTE

The “other support services” bucket — up 118 FTE — is doing most of the work and is the least transparent category. This is likely where special education aides, wellness staff, tech support, and facilities positions live.

Some of this growth is real and necessary. Special education mandates have expanded, and mental health support positions were added in response to well-documented student crises in Palo Alto. But the net picture is clear: the money went to support and admin staff, not to teachers in classrooms.

The cost of each employee has also gone up - about 86% of the budget goes to salaries and benefits — but a growing share of that is mandated pension contributions (the CalSTRS employer rate went from 8.25% to 19.10% over the past decade — more than doubling) and healthcare cost increases. These aren’t spending choices anyone in Palo Alto made. They’re structural costs that silently eat the gains before a single dollar reaches a student.

The recent conflict makes more sense through this lens. When Superintendent Don Austin proposed cutting 18 positions, it was met with fierce resistance — and he ultimately departed. The teachers’ union is requesting what the district characterizes as a 28% total compensation increase. Both sides have legitimate arguments — teachers can barely afford to live here, and the district can’t absorb $42.5M in new ongoing costs. But the deeper issue is that both are fighting over a pie that’s being consumed by forces neither controls.

Palo Alto Taxpayers (not the federal or state government) pays for all of this (and more)

Palo Alto is a “basic aid” district, meaning our funding comes almost entirely from local property taxes, the voter-approved parcel tax, and occasional bond measures — all of which have continued to grow even as enrollment shrinks.

The budget conversation usually focuses on the $340M operating budget. But Palo Alto taxpayers are also funding ~$55M a year in bond debt service for construction, plus $16.5M in parcel taxes. Yes, bonds are legally restricted to facilities — the district can’t redirect that money to salaries. But from the taxpayer’s perspective, it all comes from the same pocket. When we approve a bond measure and a parcel tax and pay rising property taxes, the total ask keeps growing. It’s worth looking at the full bill, not just one line item.

This also brings up the upcoming June 2026 Parcel Tax renewal - do taxpayers really need a $16M parcel tax alongside our $340M total budget (plus $55M construction bonds)?

The question PAUSD needs to answer is what the right staffing mix looks…

What is the “right” FTE mix for PAUSD? Should we hire more teachers and reduce support staff? How will the incremental FTE hiring improve the quality of education for our children?

You can see the full interactive budget breakdown yourself in the PAUSD Budget Explorer

Who is reading all the stuff AI is writing?

Sun, 01 Mar 2026 09:00:00 +0000

tl;dr: If it took you twenty seconds to generate, don’t expect me to spend twenty minutes reading it.

I’m seeing a massive influx of AI-generated documents: deep-research reports, meeting notes, PRDs, and strategy decks.

They look professional. They follow a clean structure. But when I push for the “why” behind a specific recommendation, the logic usually collapses.

Writing forces understanding: For me, writing isn’t about producing paragraphs, it’s about clarity of thought. The act of writing forces me to build a clear understanding of the topic. I have to grapple with the nuances and trade-offs.

That process takes time because thinking takes time. When I use AI to skip the writing, I often end up skipping the thinking, too.

No time to read: AI can spit out a professional looking, polished document in no time. However, it doesn’t reduce the time required to read or understand it.

In fact, AI docs are often harder to digest, because I don’t know what the person was actually thinking (vs what AI wrote for them). When I read a human-written doc, I’m tracking their logic. When I read an AI doc, I’m constantly wondering: “How much conviction does the writer have on this point?”

I have stopped reading documents that are purely AI-generated. If you didn’t have the time to write it, I don’t have the time to read it.

I use AI every day, including writing this blog post. But I still believe that writing is a reflection of thought. Using AI to write without thinking is just generating noise.

If you really want to send an AI-generated doc, just send the prompt.

Public vs Private School College Acceptance Rate

Sun, 01 Mar 2026 00:00:00 +0000

Comparing PAUSD with Harker: Harker doubles your top-50 odds (though there is selection bias), UCs are the most common outcome for PAUSD, and Foothill is smarter than it sounds.

I wanted to get some clarity on what the actual data looks like - especially comparing public school in Palo Alto with other private schools in the bay area. Fortunately, PAUSD publishes detailed matriculation data for both Gunn and Paly. I also found that Harker publishes similar data which gives us a good basis for comparison.

I’ve consolidated it in a quick little dashboard. A few things stood out –

A lot of PAUSD students go to Foothill College. Foothill is an excellent community college, with a very clear path to transfer to UCs. It allows students to have a smooth transition from high school to getting into an excellent university, without the pressure of getting into a top university right out of high school.

Private School (Harker) doubles your likelihood of getting into a top-50 school. About 33% of Harker graduates go to a top-50 school (compared to 17% of PAUSD). It’s also true for top-10 schools where Harker sends ~12.6% of its graduates compared to ~8.5% for Gunn and ~7.6% for Paly.

This does not account for selection bias (as Harker is a private school, it can select its students), so maybe it’s not that much of a difference in the end.

The UC system is still up there. Roughly 17–20% of PAUSD graduates go to a UC campus (across all 3 years).

You can explore the full data — by school, by university, or side-by-side — in the college tracker. Data is 2022–2024 combined and QS 2024 ranked.

AI's Impact on the Future of Software Engineering Jobs

Thu, 26 Feb 2026 10:00:00 -0800

AI is making building software cheap and accessible. This is unlocking massive productivity for everyone, but is also commoditizing software engineering. We are at the end of the software gold rush, and software engineering is going from elite to skilled trade, available to everyone.

AI is fixing the supply side of software engineering. Software Engineers are currently a luxury. Only well-funded businesses can afford them, and most others end up using generic “off the shelf” SaaS tools. These tools are often rigid, clunky, expensive and don’t always fit your workflow.

AI is fundamentally changing that. It’s making software accessible to everyone.

While there are some recent incidents on AI-driven layoffs, I don’t buy the mass unemployment narrative. In fact, with AI, there will be tons of people who can either build their own software or help others build it.

However, these will not be “premium” jobs. They will be like accountants, electricians, plumbers, etc.

Software engineers have had an incredible 30-year run of amazing pay and perks. This tech gold rush is over, and software engineering is transitioning from a luxury priesthood to a blue-collar trade. When the cost of code is $1, being a 10x engineer is a superpower. When AI drives the cost of code to $0, 10x is just another way of writing zero.

We’re about to see a massive explosion of “invisible” software—tools built for one person, one business, and one specific problem. The era of being forced into a SaaS monoculture is over. This is going to a net-positive, businesses - even small ones - can solve their own problems - and the ones that are able to leverage this superpower will have a significant edge over others.

Bringing YouTube one step closer to replacing traditional TV - Announcing VideoMirchi.com

Mon, 08 Oct 2012 20:08:00 +0000

I had subscribed to Indian programming on Dish Network, mainly to get the latest updates on bollywood movies and music. YouTube always had all the content, but never had a good way to discover it in a easy way.

This is fixed now. I am super-excited to announce Video Mirchi. You can access it at http://www.videomirchi.com.

Video Mirchi looks up the top charts every day and creates a continuously playing video channel powered by youtube. There is no login or signup - its just like a TV channel playing the latest bollywood videos.

It works on both computer and mobile. Connect it with your TV directly or via Airplay.

Enjoy!

A personal review of Lytro Camera

Mon, 05 Mar 2012 05:19:00 +0000

I just received my Lytro camera on Friday. I played with it for 2 days, carrying it almost everywhere I went. There were a lot of 'pro' reviews on the camera, but very few consumer ones, so I thought I will write down my thoughts on it.

Quick summary on Lytro as a camera:

What I liked:

It is a very well built camera. It feels great in the hands, and its construction is high quality.
Its fast. There is no shutter lag. The touch interface works great and is easy to use.
It charges via USB when connected to the computer.
Software is minimal but decent. Lytro uses its own image format, so you need the software to view the photos. There are tons of other things I wished it did, but it does the job for now.
Image quality (besides the resolution) is pretty decent.

What I didn't like:

The screen is horrible. I can't believe they put such a poor quality screen on this camera.

The demos that they have on their site are definitely not with a real camera.. the screen in their demos is crystal clear.
It has almost no viewing angle other than 'straight'. Makes it difficult to take pictures, specially because you have to rely on looking at it so much.
Its almost non-functional in sunlight.

Direct view - you can see the highly pixelated image

View when tilted by about 15 degrees. The image almost looks like a 'negative'.

The final resolution of pictures (when exported as jpeg) is 1080 × 1080, which is ok for sharing on facebook but too low for most other uses.
The camera actually requires you to take pictures that are out of focus, which can be corrected using the software. So you can't be 100% sure while taking a photo if you will get the focus you want.

Taking Photos

Lytro only has 2 buttons - Click and Power. It also has a touch-slider to zoom which is pretty responsive. This makes it easy to capture photos.

It took me a few hours to get used to the camera, especially because of the lack of a good display and mediocre preview quality. It also takes a bit of work to get used to how to pick the right focus range.

According to their tutorial, you have to take the photos so that the main subject is slightly blurred, which can then be brought into focus later using the software. I had a lot of shots where it went out of the focus range - like this sample photo from gizmodo’s review so I could not get it as sharp as I wanted. I guess this is something that will improve by practice, but its a bit tricky to take an ‘imperfect’ photo and trust the camera to make it ‘perfect’ later.

Overall speaking, I got used to most of the stuff and was able to take acceptable photos within a day. The camera literally has no 'settings', which is nice.

Living Pictures

Lytro touts this camera as ‘click now, focus later’. This is cool for marketing, but is just a way to bridge the gap with existing cameras. Current SLR cameras are quite fast, and I can see exactly what the picture looks like while taking it, so the ‘focus later’ part of lytro doesn’t seem very useful to me personally.

However, the whole concept of ’living pictures’ has added another dimension to the photographs. You can craft the picture so that the user discovers something new when he clicks on the blurred part, which really makes the photos feel ‘alive’.

For example, the picture below looks like a standard rope with blurred background. Any camera with a decent lens can do this. There is nothing that stands out. However, clicking on the background reveals a new detail that makes the photo more interesting.

This is a whole new paradigm in photography, beyond just 'decide the focus later'. It adds a new meaning to the photo which has been historically lost with traditional photos.

Focus Range

Lytro does not have an infinite focusing (or refocusing) range. While taking pictures, you have control over this ‘range’. As far as you are within this range, you can focus on any object inside it.

There are also some interesting tricks. For example, in the picture below, the frame of the mirror is on its own focus plane, but what you see inside the mirror and what’s in front of the scene is on a separate plane. Clicking inside the mirror or on the house in front of the car focuses both the regions.

Until now, we have always been dealing with Aperture, Shutter speed, etc. but the Lytro adds a new dimension which is fun to play with.

The software currently does not allow you to create a photo with infinite depth of field, but I believe that a future update will make it possible.

Low Light Performance

The camera does OK in low light, definitely not as good as my 40D at f1.8, but better than my compact powershot camera. Here is a sample:

Conclusion

Lytro a pretty solid camera as version 1.0 of a brand new technology. With the amount of data that the camera captures, it might be able to do amazing things like 3D and HDR. However, they have started with a simple use case - excited to see where it goes from here.

After using it for 2 days straight, I have already started missing the ability to click anywhere in the photos I took using regular cameras.. wondering ‘what was hidden there’.

As for now, it does not replace a regular camera, but is something fun to have and make photography more interesting.

If you are interested in photography, already own a bunch of lenses and a nice SLR, and have $400 to spare, this is going to be a very fun ride.

Here is my public stream on Lytro, have fun!

–

Btw, if you want your significant other to get you one, let them know by adding it to your wishlist at theicebreak.com.

Comments

Paul Lloyd (March 05, 2012):

Cheers buddy! I also received my Lytro camera yesterday. I’ve waited for this camera since long time. The camera quality is very good. Thanks

Aniket (March 05, 2012):

Thanks for your views as a consumer/user. Given these views, it would be very interesting to understand your perspective about the product and/or company as an entrepreneur/investor.

Would you have invested in them; say today?

Dwipal (March 05, 2012):

The camera takes advantage of all the advancements that have happened in computing, like cheap and fast storage, faster processing, etc. Even if someone wanted to do this 5 years ago, it wouldn’t have been possible then.

There is a lot more to this technology than just ‘focus’, so I am quite optimistic on it. It all depends on how well they can execute, and whether they can deliver this promise in the long run.

Regarding investing, just looking at their technology, I would have invested ( I would’ve invested in them even 2 years ago if I had a chance to - its the kind of product I like to follow ).

Girish (August 11, 2012):

Thanks !!!

www.noncamerasmartphones.com

Dr.Murat Enoz, MD - ENT Doctor (February 18, 2013):

Focuses on the location you clicked … Gorgeous Greetings Burun Estetigi

dimz (August 28, 2013):

Thanks for providing such a great article, it was excellent and very informative.

thank you :)

Smart TV FAIL. Let's go back to the Idiot Box.

Tue, 14 Feb 2012 21:00:00 +0000

Having observed the whole ‘connected tv’ phenomenon first hand since my early days at YouTube, contributing to XBMC since its inception and being really interested in this space personally, I am very upset with the direction where Smart TV is going.

I think that the future should not include 'Apps on TV', 'Browser on TV', and its also not 'Read Facebook or Check Email on the TV'. They are terrible to use and there are some fundamental reasons why they will never be good:

TV is a consumption device for media (photos, videos and music).
An average TV is a few feet away from the user. This makes it difficult to read any text that is not blown up to the size of subtitles.
It is also a shared device, so anything that is relatively personal (like email) is not practical to use on the TV.
The interface on TV is simple: Big menu items and Big boxes. Anything beyond this gets cluttered up because of its size restrictions.
It is really awesome at what it does: Play media. It sucks at almost everything else.

Now, if you look at the trends that are happening (for TV space):

Facebook / Email / Twitter / Browser on TV
Big fancy keyboards connected to TV
Reading news on TV
Searching for content on TV
Different apps on TV

All these are enabled by connecting the TV to the internet. TVs have become more smarter in the last 5 years than they have ever been, and everyone is itching to get on board with these new capabilities.

However, I think that loading TV with all these features is going to fail.

An ideal TV experience should be:

Use TV as a dumb device to play content from any source - brodcast, internet or somewhere in between.
Give it a minimal interface so its easy to play this content, like play / pause / next.
Use an additional device (Laptop, Phone, Tablet, etc) to control the programming.

This way, next time you want to watch Netflix or listen to Pandora, you can go Netflix.com (or open its app on your smartphone) and select 'Play on TV'. No dealing with confusing interfaces, no app crashes on the TV.

However, this is not happening due to a variety of reasons:

There are no 'standards' around this. I cannot build a TV that supports standard X, which will enable it to play content from different sources directly without writing native apps.
A TV without interface doesn't look 'sexy'. You can't show-off that it has 'no interface'. Except if you are apple :).
The TV will need a companion device which actually has the interface. It can be a tablet, laptop, phone or even the source / website itself.

Imagine if we had to have apps for every channel that we have subscribed to on your Cable or Dish network, all with a different UI. This is where Smart TV is going right now.

Apple is the only company that has done at-least something on these lines, via AirTunes and now AirPlay that is directly integrated in iOS. Its still a bit clunky to use, but definitely a step in the right direction.

In the end, the primary role of a TV is to play content, and trying to make them ‘Smart’ is like putting lipstick on a pig.

Hopefully we will see a TV that has no interface, can be fully controlled via my tablet, and can play just about anything from everywhere.

Comments

None (June 17, 2012):

www.noncamerasmartphones.blogspot.com

Phone Guru (September 01, 2012):

Thanks a lot !!!!

www.noncamerasmartphones.com

None (August 31, 2013):

The Rocket Droid Pc is a device that is aiming to be the most powerful portable mini Android dongle. It will plug directly into the HDMI port of any TV and instantly turn it into kick-butt SMART TV - bringing all the power of an Android Mobile Phone to your previously boring TV! SMART TV

Smarter Email

Wed, 08 Feb 2012 06:42:00 +0000

I love Email.

It just works. It works because it is flexible, open, and anyone can use it. It works very well in heterogeneous environments as the email standard is well documented, accepted and implemented.

I have looked at a ton of ‘solutions’ that try to replace emails, but nothing really works as seamlessly as good-old email.

The goal of email is to enable communication and think that it can be improved. It will be stupid to completely replace this amazing means of communication, when it fundamentally works well. What needs to be done is to make it ‘smarter’.

What's broken is not the ability of emails to enable communication, but that the forms of communication have now become more complicated whereas email is still at its original design built for person-to-person messaging. Because email worked for everything, everything piled onto it making it complicated.

What needs to happen is that a standard should be created on top of email, that can handle ‘special’ use cases which can then be implemented by clients in a cross-platform way.

A few examples of this are:

Messages from discussion groups can be similar to RSS feeds. They are usually read-only and rarely need to be responded to.
Special handlers to 'automatic' messages, like calendar invites, flight alerts, shipping confirmation, etc. They don't need to be shown like a 'message', just need to get to my mailbox so that I can access them if I need to.
Notification emails should show up as, well, notifications.
A good way to handle things like groupons, which, even though come in as email, aren't really messages.

There is no one company that can 'fix' this, as any change to email needs to be standardized. However, small steps can definitely be taken to create email formats that are 'machine readable' by the client so they can be organized properly.

A first step in this direction is to have a major email provider (eg. gmail) to start supporting ‘smart email’ formats. This will make the services to send emails compatible with this format, which can then grow and expand to more and more services.

I would love to see some work done here, and see someone solve this problem. May be its time for someone to start a new email company :)

Comments

None (February 15, 2012):

This is 100% spot on. I’ve thought about this a great deal in the past, and discussed it with some folks.

Thing is, you don’t even need a new standard, since the mechanism is already there. All we would need is for folks to agree (that may be harder) on the use of standardized x-headers. Then clients can implement “smart” filtering, etc based on the presence and value of them.

For example (none of these should be considered well-thought-out ;-) x-from-company = Netflix Can be used to filter on company

x-is-billing = True Can be used to route to a folder of bills

x-period = Monthly Can be used to determine subscription or periodic freq

If you put all those together, you can create a folder for my Netflix account statements and highlight it when a bill is near or past due. You could allow a search fo all bills this month. You could do quite a lot.

Dwipal (February 15, 2012):

Yes, that is exactly what I am talking about! All the pieces to do this are already in place, people just need to agree on it.

Aniket (February 15, 2012):

Didnt Wave try to re-invent email - and faced a lot of inertia from users? Didnt reach critical mass. Although I understand what you are proposing is quite different from Wave’s propositions - but the case in point is that people (users at large) shed inertia really slowly - or just dont.

GMail labs also from time to time have some interesting little featurettes. For example - they mark email that is addressed directly to you as important; etc… I believe that such flexibility is best left to individual users to tune their mailboxes to their preferences (rather than standardize). Folders and (perhaps smarter) filters serve the purpose in most cases.

Dwipal (February 15, 2012):

Wave tried to replace email with something very clunky. It didn’t really reduce the ’noise’ of email, or add a good structure to it. Rather, it was really noisy. It also tried to do too much.. like editing documents, which is not meant for doing inside of email.

Jalpesh (February 18, 2012):

On similar lines, may be write extensions for desktop clients like outlook. show interests in people using it and hopefully one of the companies will add this in their apps. I see more and more web mails are mimicking desktop app functionality (apart from their own unique features) to move desktop features on thinner client. I think it’s difficult for gmail or Microsoft or yahoo to make these standards esp because they have to interoperate

The Native-App economy and its future

Sun, 02 Oct 2011 17:39:00 +0000

Since the launch of iPhone, everyone has been trying to replicate the App Store model that Apple created. Every operator, OEM and even independent companies are trying to create their own app stores and native app platforms.

The app store/native app model is inherently a walled garden model. This is analogous to AOL in the early days of the internet. AOL was the first one to truly bring internet to the ‘masses’. Internet was very young at the time, and was difficult to use. AOL, by controlling the ecosystem and adding restrictions so that everything works well within it, was able to drive adoption for the internet.

In 1999, DoCoMo launched iMode in Japan and did what AOL did - but to mobile devices. All of a sudden, people could open emails, look up stock prices, check the weather and even read magazines all from their phone. iMode was extremely popular and put Japan way ahead of the world in mobile use. Following DoCoMo, other Japanese operators also created their own ‘private ecosystems’ and achieved considerable success.

The common theme between all of them is that they created a very strong value proposition by productizing new, difficult to use technology.

However, as time went on, these walled gardens became barriers and hampered innovation in the space.

The technology matured and people wanted ‘more’ out of it. They started to created services that ran independently, and the ‘walls’ started to crumble.

It also got difficult for publishers to time to build services and innovate on business models that weren’t a part of this ‘walled garden’. Slowly but surely, things started moving to the ‘open internet’ and even though the initial experience was not as good as the walled garden, both consumers and publishers embraced it.

I believe that the Native Apps fall under the same category. With the launch of iPhone, smartphones finally became powerful enough to be able to do things that ‘common people’ would want. However, because the technology was still developing, the only way to create a great experience was to go ’native’ and build a proprietary app. Apple was able to use this to their advantage and create the app store.

With HTML5 developing at an amazing speed, more and more people will start to see the clear advantages of building on an open platform. Its a massive waste of effort having to write native apps on different mobile operating systems that run on hardware of pretty much the same capability.

There are already companies that have started to work in this space, allowing users to get the best of both worlds. The technology is still young, but extremely promising, and a couple of years down the line, the notion of ‘you must build your app for 5 different platforms’ will be a thing of the past, just like AOL. There will still be some native apps, like we have them on desktop/laptop but they will be extremely specialized.

App stores - as a way to discover ‘apps’ might always stay on but ’native’ as a platform should start counting its days. Until then, its the age of the ’native app’.

Photo credit: http://sethmurphy.com/node/8

2 Year Anniversary

Fri, 30 Sep 2011 20:10:00 +0000

Next week, I completed 2 years of leaving my amazing job at YouTube to enter the world of startups.

Looking back, there is a ton of things that happened in these 2 years, including starting 2 companies, having a baby, traveling, fundraising, investing, advising, volunteering, etc.

There is a lot that I have learned from this rollercoaster ride, and its now time to reflect on some of the key things

Dont’s

Do not leave a good job unless you know what you are going to do
Do not start a company without doing enough research
Do not think that you can go alone in building a company

Dos

Do make sure that you have enough savings to last a while
Do find co-founders or have a founding team that you have a good working chemistry with
Do have family time, startup is a marathon not a sprint

Thanks to my wife for understanding and keeping sanity in some of the insane times.

Want to know your users better? Run your own analytics!

Thu, 29 Sep 2011 07:12:00 +0000

(Originally posted on theicebreak blog.)

Theicebreak is all about understanding relationships. It makes sense then that we would want to ensure that we have a strong understanding of who we are in a relationship with - our user base.

To do so, we really focus on tracking every feature and functionality through cohort analysis, as championed by Fred Wilson, Dave McClure, Eric Ries’s lean startup movement and others.

There are now a couple of services that provide easy tools to do so for startups, including everyone’s favorites…Mixpanel and KISSMetrics. We tried them all, but immediately hit some crucial limitations of these online platforms:

We have multiple interfaces (web, mobile, api) and we would have to individually integrate with analytics on each one. A way to solve this is to integrate on the server side.
We wanted to combine this data with other attributes, for example, break down a certain feature by male vs female users, or married vs engaged couples.
We needed to be able to run custom queries and reports based on the data in a simple way.
This data is private and very important for the company, and we would like to 'own' it.
It should be inexpensive :)

Most of the analytics services (except the really expensive/complex ones) are typically focused on implementing the tracking in javascript or other front-end APIs. Some of them do support server side logging, but it is limited.

The only way we could meet the above requirements is to build our own system. Once we started to write it, it actually turned out to be quite easier than we originally thought, and also a lot more powerful.

The key requirements (besides solving the problems I described above) were:

It should be quick to build and simple to use.
It should be re-usable, and possible to run lots of complex analyses with ease.
It shouldn't take any more effort to log different action types as the site grows.

We created a very simple 'events' table called event_log (we use mysql). This table basically had 5 fields:

uid
event_type

event_action_id

time_created

description

Any action that a user takes, like answering an icebreaker, changing the password, posting a moment, updating stats, etc. gets an entry in this table. We also store the event_action_id that points to the actual action. This way we can dig deeper if needed. We focus on actual ‘actions’, not page views.

This is done via a simple api call to log an entry to this table after the user performs an action. This has been extremely powerful for us, and forms the basis of most of our decision making reports.

Because the table is simple but contains all the necessary fields, we can now run complex cohorts as well as simple reports just off of this table. There can be joins and other analyses that can be very easily executed without too much overhead. It also really simplifies the task of getting the same data on different types of actions, as it’s all normalized in a clean, single table.

As an example, a basic cohort table can be easily generated by a query similar to:

SELECT
week(u.time_created) AS cohort,

week(g.time_created) AS period,

count(*) FROM event_log g

JOIN user u ON u.uid=g.uid

WHERE

u.time_created > ‘2011-01-01’ AND

g.event_type = 1

GROUP BY cohort, period

ORDER BY cohort, period

A quick post-processing of the results and you can generate some very useful charts out of it; below is an example simulation of how retention would look for a typical website for a few weeks following a launch-spike.

Adding a simple dropdown for the event type, and you can get the same chart broken down by events. You can also very easily add other filters by doing joins with other tables and get real insights on who your users are.

For example, we can run reports like:

What's the usage of the icebreaker feature among married males?
What's the overall retention broken down by each relationship type (i.e. married, dating, etc)?
Do males add more stats or females, and how do these numbers change over time?

This is a good initial approach; it provides a lot more flexibility than other services, and will work effectively until you hit a few million users. Then you will start having 'scaling issues' (a good problem to have).

There are definitely some drawbacks to this approach, like:

The events table can grow really large. We only track the 'important' events, and for the unimportant ones, we track it at intervals (for example, we only log one visit per day). You can also vertically split the table to keep the size under control.
This is not connected to front-end analytics; doing so might complicate the system.
You have to write this from scratch. Even though it was relatively easy for us to write, it's still more effort than adding 3 lines of javascript code.
It can get slow... but it's not a user-facing feature, and we can cache the results. We also run this on a replica database, so it doesn't really affect the performance of our live service.

We use these cohort reports in conjunction with regular analytics (getclicky and Google analytics) for traditional page views, visitors, funnels, campaigns, and conversions. This has helped us understand a ton about how the site is being used, and what our users want and are doing.

Cheers!

Dwipal

http://blog.theicebreak.com/blog/2011/9/29/want-to-know-your-users-better-run-your-own-analytics.html

Spotify, MOG, Rdio, Rhapsody for Bollywood Music

Wed, 27 Jul 2011 17:16:00 +0000

I now have at-least 6 different devices that are used to play music, so I think its high time to look seriously at streaming music services.

I tried Spotify, Rdio, MOG and Rhapsody. All of them seem to have good audio quality, and a pretty solid music catalog for western music.

My requirements are simple:

Must have:

Good Bollywood music catalog
Share accounts, so that both me and my wife can listen to music at work
Android, iPhone, PC support

Nice to have:

Offline playback
Upload songs already owned by me

	Spotify	MOG	Rdio	Rhapsody
Bollywood Catalog	Yes	Yes	Lacking	Lacking
Shared Account	No	Yes	Didn't Check	Didn't Check
Android, iPhone, PC	Yes	Yes	Didn't Check	Didn't Check

To test the bollywood catalog, I used following combination of music:

	Spotify	MOG	Rdio	Rhapsody
Zindagi Na Milegi Dubara (2011)	Yes	Yes	No	No
Delhi Belly (2011)	No	No	No	No
Sangam (1964)	Yes	Yes	Yes	Yes
Purani jeans (Song)	Yes	Yes	Yes	Yes
Patiala house (2010)	Yes	Yes	Yes	Yes
Double Dhamaal (2011)	Yes	Yes	No	No

Based on my requirements, MOG is a clear winner. It has (almost) the full bollywood catalog. I also like some of its other features:

MOG Radio
Public playlists
No desktop download required
Available on a wide variety of devices, like Sonos, Roku, etc.

Its UI is a bit strangely designed, but should fine once I get used to it :)

Go MOGging :)

Comments

None (October 30, 2011):

thanks this is exactly compare i was looking for..

None (June 25, 2012):

Have you checked www.saavn.com ? both spotify and mog dont have the latest bollywood albums whereas saavn does. saavn has a nice iphone app too.

None (June 21, 2013):

BollywoodFind All latest bollywood news,Bollywood Wallpapers,Bollywood Hot and latest Gossips only on bollywoodjosh.com

None (November 29, 2013):

Hello there, just became alert to your blog through Google, and found that it’s really informative. I am gonna watch out for brussels. I will appreciate if you continue this in future. A lot of people will be benefited from your writing. Cheers!

My web site :: Web site

None (May 31, 2014):

3 devices for music . waooo. nice admin, i have 2 only pc and lappy :P with best whished: Nimra Yasmeen http://mp3videosongs123.blogspot.com

The 'social graph' lock-in is overvalued

Thu, 30 Jun 2011 15:36:00 +0000

We keep reading that the biggest asset Facebook has is their social graph, the network of friends and friends of friends that is close to impossible for other companies to replicate.

Facebook initially grew their network not just because people wanted to be friends with others on an online service, but because it provided a real value that people wanted to capture (according to my friend who was at school when facebook was getting big, that value was to get invited to all the parties, or spy on a girl).

If a service provides enough value for people, they will go through any amounts of pain to sign up to capture that. Think of dating sites like eharmony and match, they are extremely boring to sign up with forms that run for a mile, but loads of people still sign up.

I had attended the summit series conference few months ago, and they had their own special social network which you could sign up. The had very long forms, no facebook connect and even no email notifications. However, everybody still signed up and it was a very vibrant service during the conference… it was exclusive and it was useful.

Google plus, though a bit too early to say, is similar. I just signed up and found enough value because of the easier privacy controls (and also the photo syncing feature).. if this does well, my ‘social graph’ will take no time in moving to google from facebook.

At the end of the day, if a service has a real use for people, it will not be that difficult for them

to get people on board. If the focus on a bottom-up approach on focusing on the value, people will sign up. In today’s world where everything spreads so fast, the social graph will take no time to move with users.

Comments

Rahul Lakhani (February 15, 2012):

I agree with the thesis of the argument. However the problem is that Facebook also knows this and they will do everything in their power to make it harder for you to move. For example, Facebook will probably never allow users to carry out its “sharing history”. Without the sharing history, I will find it hard to move to other networks. Other social services will have to exploit a niche and grow from there. For example, Google Plus is probably not betting on becoming the next Facebook. It will still achieve its purpose as #2 or #3, by capturing your social activity and sprucing up its own search/ads results.

Dwipal (February 15, 2012):

I think there will be ‘parallel networks’ that will build along with facebook, but they will eventually start cutting into facebook’s core. I am already seeing pinterest and tumblr doing that, there are communities of people directly connected making facebook less relevant as you use them more and more.

Bitcoins!

Mon, 27 Jun 2011 17:22:00 +0000

Have been playing around with bitcoins lately. Even though I seriously doubt the long-term viability of such a currency, I think its pretty amazing to be able to use it right now.

If you haven’t heard about it, Bitcoin is a P2P, completely decentralized currency. Its like a digital version of having cash, with all the benifits (and disadvantages) of cash. Its anonymous and can’t really be tracked (just like cash), but can also get stolen if someone gets access to your files.

People have been using it to buy t-shirts and accesories, server space and a physical NY restaurant even accepts payments using bitcoins. Imagine traveling the world without worrying about currency exchange problems and having a single currency that works everywhere.

To get started, check out bitcoin.org or download the app. on your computer and send me your bitcoin address so I can send you some!. If you are feeling rather generous, my address is 1JcNMmEUYCieADecgUNkjE3vYTanJhD9wu :)

Dwipal

Comments

zameer (December 01, 2021):

Karnataka 562123 080736 70915 SHREE SAI PACKERS AND MOVERS #18/3, 30ft road,, sri krishna colony, kuduregere main road,, alur post, Bangalore, Packers and movers bangalore

Guru TV Repair 52 shampura main road, Munni veerappa layout Kaval bairasandra Bengaluru, Karnataka 560032 080888 33910 Led tv repair in bangalore

Graphica Gauges and Tools S. No. 37, Ramnagar, Ramnagar Tempo Chowk,Vadgoensheri Pune, Maharashtra 411014 098508 84992 Thread Ring Gauges

iRobo Wellness Private Limited AryaHub Mall No. 22 Hope farm Junction Whitefield Bangalore, Karnataka 560066 099806 58125 Massage chair

In search of a wireless keyboard...

Sun, 05 Jun 2011 04:53:00 +0000

So one day, I decided that I want to go wireless with my keyboard and mouse. The main reason was that I didn't want to plug in the USB cable every time I get into the office.

After searching for good, full sized bluetooth keyboards, I soon realized that there is nothing good out there. So I decided to get the Logitech MK710

Where is my digital magazine?

Mon, 18 Apr 2011 05:38:00 +0000

With the millions of new blog posts, news articles, tweets and updates posted daily, getting to the content we want in real time is easier than ever. The only caveat: we need to know what we want.

Traditionally, we have relied on the major publishers and distributors to feed us the content in the form of newspapers, magazines, journals, books, etc. The publishers would decide what content would ‘sell’, and will then ‘productize’ it so people can get access to it.

This model worked really well for over a century. However, there was one big inefficiency with this model: The content creator didn’t really get much in terms of control or revenue. In fact, for most cases, less than 15% of the revenue of a book goes to the author. The publishers and distributors were able to pull this off because they basically controlled the supply chain.

Now, with the power of the ‘internets’, anybody can create a piece of content and make it accessible to users in the same way as a big publisher does. So there are a million new pieces of content created every day, ready to be consumed.Of all the awesomeness of this new model, one thing that is terribly broken is the part where publishers curate the content to make sure that it is of a certain quality before it reaches the users.

In the publishing industry, there are people dedicated to reviewing thousands of articles, book ideas and other content before something gets published. Because of the limited space available in physical media, they have to be quite picky what gets included and what doesn’t.

I think that there is a massive opportunity right now to do this with online content. I have been relying on my social graph to get interesting content daily, but its still too much content and too little variety. What is needed is for someone to create a special ‘magazine’ on the internet that is both personalized for my taste, but also contains enough variety so that I can ‘stumble’ across interesting topics.

I highly doubt that something like this can be completely crowd sourced. It will have to be a hybrid between a person collecting interesting content along with stuff that is popular. There is a dire need of a ‘human touch’ here - a curator for online content.

Unless someone does this right, it is going to be difficult to justify its value. However, the time is ripe for someone to create a ‘Facebook of Content’.. a place where everyone goes to get the latest news customized for their taste, but also containing enough variety :).

Till this happens, back to twitter + facebook + google news + google reader + timesofindia.

Dwipal

Photo credit: http://www.flickr.com/photos/vincentgallegos/4562911007/

Enforcing honest business practices in a 'Flat World'

Fri, 04 Mar 2011 19:02:00 +0000

Last year, when my parents were visiting the US, I decided to get Indian TV programming so they can watch it here. As Dish Network bill can go above $100 per month with such programming, I decided to go the Internet route by signing up with a service called WatchIndia.tv.

WatchIndia streams TV channels directly from India (including the ads, which is nice) over the internet. Its a subscription based service with monthly/yearly subscriptions. I purchased their 1 year service package.

Within a few weeks of getting the service, their quality started going down. It will constantly buffer or not work at all, and was overall very slow.The worst happened in december, when the set top box stopped working, and I was without indian programming for over a month during holidays.

Their customer support was polite but not helpful at all, and after a lot of arguing, they fixed my box and told me that I will get 3 months of extension to my service. My service would now expire in June instead of March.

Yesterday, I suddenly received an email from them saying they have 'renewed' my subscription for the next year and charged my credit card. I immediately called them to cancel it and refund the charge but they refused to do so without major fees. There was no notification that my subscription was going to get renewed, and I had not signed up for auto-renewal in the first place.

Now, the thing about WatchIndia is, even though they are focused primarily on US consumers, they have no US presence. There is no US office address or contact information (except a toll free customer support number). The only recourse I have is to dispute the charge on credit card, which I can luckily do. Basically, the company used typical sleazy tactics to renew my account and refusing to cancel the subscription.

This brings a question to the fact that as more of international companies start selling their services in the US, what recourse do the US consumers have so that they follow the law. Most states in the US have strict guidelines on companies that subscribe people for auto-renewal on their credit cards, and you can file a complain at the BBB in case they break the laws. However, international companies might not follow these guidelines, and its very difficult to actually go after them if they break the rules.

Hopefully, as more of these services start getting traction, the legal system will evolve to keep them from cheating their customers. Until then, we just need to be careful :).

VIM 7.3 with Persistent Undo

Fri, 04 Mar 2011 17:43:00 +0000

VIM 7.3 recently got released, and supports persistent undo!

This post describes a simple way to configure it. You have to compile it / get it from unstable repo, but totally worth it :)

Enjoy!

TheIceBreak Tech Overview

Tue, 15 Feb 2011 07:40:00 +0000

A few people asked me what we use at TheIceBreak, so here it goes:

Linux / Apache / MySQL / Python
Pylons Framework (SQLAlchemy ORM, Mako Templating Engine)
Lucene for Search
JQuery for website
Appcelerator Titanium for mobile app.
Redis for caching
GIT for source control, hosted on github
nginx as load balancer
Linode VPS for hosting webservers / database
Amazon S3 for images

Besides this, it consists of:

3 main servers: Web, Search, Database
1 dev server, which also contains a database replica (in case the main server crashes)
The dev server also hosts a 'stage2', which is as similar in structure to the live site as it can. Everything that is pushed live goes through stage2 to make sure that nothing breaks.

Using this setup, I have been able to keep the latency of most (dynamic) pages to under 150 ms.

On Leadership

Thu, 10 Feb 2011 08:01:00 +0000

I recently watched an excellent episode of Fareed Zakaria GPS on the topic 'What makes a good leader' that was sitting on my DVR since months. He had some of the biggest leaders in business and politics (Lou Gerstner, Tony Blair, Admiral Mike Mullen and others) talk about their take on Leadership.

I thought I would summarize what were the biggest tasks that a Leader should accomplish:

Set up a clear long-term vision and guidelines for the company.
Make sure that the execution follows this long term vision / guidelines. People are free to do whatever they want as long as it helps the company reach its vision.
Be a good listener, you will need all the data you can gather to come up with a good vision
True leaders will show their strength during a downturn, its easy for anyone to do well when the company has a strong momentum for growth.

Even though these points can easily be found in any leadership/101 class, its amazing how many companies and products fail at the basic task of having a clear vision and eventually loose their significance.

I have had first hand experience with all the above points, and can’t agree more with all of them. The last point was particularly interesting that you can identify a true leader only during a downturn. If you hire someone when your company has a lot of momentum and is growing well, you don’t need a lot of leadership skills to run it (too much leadership might actually disrupt the growth). True leaders can really be identified clearly during a downturn.

Finally, a great quote by Eisenhower: “Leadership is the art of getting someone else to do something you want done because he wants to do it”.