LinkLog: Some Good Advice on Idea Generation

I am loving James blog (spent an hour this morning reading random entries after discovering it). He has some good advice on idea generation:

Get good at idea generation. 

Ideas have a bad reputation. They’re a dime a dozen. They’re worthless unless implemented. Success is 90% perspiration. We’ve all seen the calls for help from a self-proclaimed designer and his business partner who have a brilliant company logo and a sure-fire concept for an app. All they need is a programmer or two to make it happen, and we all know why it won’t work out. 

Now get past ideas needing to be on a grand scale–the vision for an entire project–and think smaller. You’ve got a UI screen that’s confusing. You have something non-trivial to teach users and there’s no manual. The number of tweakable options is getting out of hand. Any of the problems that come up dozens of times while building anything. 

As you read through the rest of the post, you start realizing that ideas are really starting points for doing something useful. Ideas trigger other ideas. This chain reaction is what gets you to start building useful stuff. Taking small ideas, thinking a lot about them, choosing alternatives and testing a few out through implementation is part of the creative process. In fact this is the skill every product team needs, anyway.


Posted in Ideas, Creativity, Innovation | Leave a comment

LinkLog: What is a Learning Management System (LMS)?



from the kind folk at MicroAssist  - A Slideshare presentation on LMS Basics.

Posted in Learning and Teaching, LinkLog | Leave a comment

Data Jujitsu – A Pragmatic Approach To Applying Data Science

Notes from Data Jujitsu - The Art of Turning Data into Product by Dr. DJ Patil. It is free and a PDF version can be found here.


This is a great resource, if you are just starting to think about applying Data Science in your organization.  I like DJ’s incremental and very pragmatic approach to applying data science to your business and building a product. He has done it at LinkedIn (in fact the first time I heard the term was from his LinkedIn posts).

Here are a few excerpts from the book which will give you a sense of what he covers. It is a selection of fragments I marked up while reading the book. It is only about 16 pages so you should certainly give it a try if you are interested in the topic.

On Data Scientists

Smart Data Scientists don’t just solve big, hard problems; they also have an instinct for making big problems small.

He proposes a very practical approach to solving problems.

Solve a simple piece that shows you whether there’s an interest.

In this lean startup world, there is a similar approach, known as building an MVP (a minimum viable product) to assess whether there is interest and validate the assumption we make about the needs of users.

On learning about problems his advice is to use humans initially and cites the example of collaborative filters.

By using humans to solve problems initially, we can a great deal about the problem at a very low cost.

The collaborative filter is a great example of starting with a simple product that becomes a more complex system later, once you know that it works.

How do you create engagement and revenue with your data product? Some examples from his LinkedIn experience.

Giving data back to user creates additional value. By creating data back to the user, you can create both engagement and revenue.

Focus o actionability of data.

“Inverse interaction law” applies to most users: The more data you present, the less interaction. The best way to avoid data vomit is to focus on actionability of data. That is what action do you want the user to take?

Putting Data Jujitsu into practice.

Data Jujitsu embraces the notion of minimum viable product and the simplest thing that could possibly work.

My favorite part is the advice to product builders.

With all products, you should ask yourself three questions:

1. What do you want the user to take away from this product.

2. What action do you want the user to take because of the product?

3. How should the user feel during and after using your product.

If your product is successful you will have plenty of time to play with complex machine learning algorithms, large computing clusters in the cloud…

It is 16 pages of practical wisdom that comes out running data science teams and building products at LinkedIn. The wonderful gesture of  sharing his experience with us in a free eminently readable report,  deserves a salute.


Posted in BookLog, Data Science - Big Data, Data Analytics and Related Topics | Leave a comment

Book: What Is Data Science?

I just finished reading the book What is Data Science?.


It is a small book (25 pages) and one of the many good starting points to learn about Data Science. This not a review but a few quotes from the book:

  • According to Mike Driscoll(@dataspora), statistics is the “grammar of data science.”
  • According to Martin Wattenberg (@wattenberg, founder of Flowing Media), visualization is key to data conditioning: if you want to find out just how bad your data is, try plotting it.
  • Making data tell its story isn’t just a matter of presenting results; it involves making connections, then going back to other data sources to verify them.
  • Data science requires skills ranging from traditional computer science to mathematics to art.
  • According to DJ Patil,  (@dpatil), the best data scientists tend to be “hard scientists,” particularly physicists, rather than computer science majors. Physicists have a strong mathematical background, computing skills, and come from a discipline in which survival depends on getting the most from the data. They have to think about the big picture, the big problem. When you’ve just spent a lot of grant money generating data, you can’t just throw the data out if it isn’t as clean as you’d like. You have to make it tell its story. You need some creativity for when the story the data is telling isn’t what you think it’s telling.
  • What Patil calls “data jiujitsu”—using smaller auxiliary problems to solve a large, difficult problem that appears intractable (he has a book on Data Jujitsu)
  • Patil’s first flippant answer to “what kind of person are you looking for when you hire a data scientist?” was “someone you would start a company with.”
  • Data scientists combine entrepreneurship with patience, the willingness to build data products incrementally, the ability to explore, and the ability to iterate over a solution. They are inherently interdiscplinary. They can tackle all aspects of a problem, from initial data collection and data conditioning to drawing conclusions. They can think outside the box to come up with new ways to view the problem, or to work with very broadly defined problems: “here’s a lot of data, what can you make from it?”
  • The future belongs to the companies who figure out how to collect and use data successfully. Google, Amazon, Facebook, and LinkedIn have all tapped into their datastreams and made that the core of their success. They were the vanguard, but newer companies like are following their path.
  • The part of Hal Varian’s quote that nobody remembers says it all: “The ability to take data—to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it—that’s going to be a hugely important skill in the next decades.”

This graphic from BigData Startups shows that lots of organizations still do not understand Big Data and predicts a shortage f 140k-190k big data scientists and 1.5M big data managers in USA alone by 2018.


I am reading a bunch of books and will probably do more of these posts. BTW, big data is not always about big data. It is an umbrella term to cover different areas that deal with deriving value out of data.


Posted in BookLog, Data Science - Big Data, Data Analytics and Related Topics | 2 Comments

Guardians Data Blog – How it Came About

From Guardian’s Data Blog:

We are drowning in information. The web has given us access to data we would never have found before, from specialist datasets to macroeconomic minutiae. But, look for the simplest fact or statistic and Google will present a million contradictory ones. Where’s the best place to start?

That’s how this blog came about. Everyday we work with datasets from around the world. We have had to check this data and make sure it’s the best we can get, from the most credible sources. But then it lives for the moment of the paper’s publication and afterward disappears into a hard drive, rarely to emerge again before updating a year later.



Posted in Ideas, Creativity, Innovation | Leave a comment

Love The Motivation Behind This Book

I really like this para – The Motivation Behind the Book Doing Data Science:

The world is opening up with possibilities for people who are quantitatively minded and interested in putting their brains to work to solve the world’s problems. I consider it my goal to help these students to become critical thinkers, creative solvers of problems (even those that have not yet been identified), and curious question askers. While I myself may never build a mathematical model that is a piece of the cure for cancer, or identifies the underlying mystery of autism, or that prevents terrorist attacks, I like to think that I’m doing my part by teaching students who might one day do these things. And by writing this book, I’m expanding my reach to an even wider audience of data scientists who I hope will be inspired by this book, or learn tools in it, to make the world better and not worse.

The solutions to all the world’s problems may not lie in data and technology—and in fact, the mark of a good data scientist is someone who can identify problems that can be solved with data and is well-versed in the tools of modeling and code. But I do believe that interdisciplinary teams of people that include a data-savvy, quantitatively minded, coding-literate problem-solver (let’s call that person a “data scientist”) could go a long way.

Posted in BookLog, Data Science - Big Data, Data Analytics and Related Topics | 1 Comment

Good Reads: Single Most Important Habit that Shaped up my Career

Kunal Jain on Single habit that defined the trajectory of my career

What is the single most important habit that shaped up my career? This is the habit which propelled me from from being just an ordinary analyst to some one who can influence, manage and mentor people in Analytics industry.

Here is the habit:

Spend a defined fraction of your day working on the the most important project / problem you have.

Please note the importance of two words here: defined and most important. You need to fix what fraction of your time you would spend and what is the most important task for you.


I discovered Kunal through an article on KDNuggets. Found his Twitter account and followed him and from there to his LinkedIn account to this article. It is nice to see people sharing so much of their knowledge through Tweets and blog posts.

A couple of other useful links if you are interested in Analytics from Kunal

Must read books and blogs on Web Analytics

Analytics Vidhya Twitter Account

Thanks Kunal. We need more people like you.

Posted in Ideas, Creativity, Innovation | Leave a comment