Posts tagged: Twitter

Twitter is better

A little while ago, I wrote about my pet peeves to do with twitter, and while they probably didn’t read my specific ramblings, they have certainly addressed my key concerns.

My biggest concern was of course, about security and twitter-apps. A little while ago, I noticed that this has been resolved. Twitter is now linked with applications in a more security conscious way. I love the way twitter now asks if an application should be  authorised to access information. Yay! No more giving my twitter account details to third party websites.

I also covered an issue that I had with grouping users to see relevant tweets together. This has also been resolved with the use of lists. In fact, lists have changed how twitter works to an extent. There is a blog post about how lists will impact follower counts. Lists provide a powerful mechanism to follow a group of people – now, if only I could have an option to see all the tweets made my the people that I am following as well as the lists – that would be cool. It probably should’t be the default, but an option to do that would be useful.

I also like the new feature where it tells you that there are new posts since I last viewed a timeline – saves me from having to click reload randomly to see if there are no posts. The little grey line differentiating the new posts help in that I know how far down I have to read to see just the new posts… 😀

Making Twitter Better

I think that twitter is a fantastic service and has a bright future. However, like a lot of new things, the question of whether it will flourish or perish is really all down how the growth is managed, planned and executed.

I should point out that I don’t know the people at twitter at all and is very much an outsiders opinion. I have been running a business for about nine years, and while it is of nowhere near the success of twitter, I’ve definitely learned some hard lessons. I am not complaining – I am however, voicing some ideas on how things could be made better.

My experience also includes working very closely with megabus.com, which grew from a fledgling website 6 years ago to what it is today servicing over a 100,000 visitors every day.

My gut instinct about Twitter is that the guys and gals are working hard to delivery one really good service really well. However, it is of a size now where service delivery should be happening in the background with little or no effort.

When megabus.com first launched and over the first couple of years, we spent a lot of time managing the hardware, software and processes till we got it right. It went through a dramatic re-architecture in 2005 and since then, the management time has dropped dramatically.

To take twitter to the next level so that it can be bigger than facebook, in my opinion, requires twitter to a lot of things:

Reliability & Performance

I don’t know the architecture / infrastructure of twitter but having used it fairly heavily over the last few days, have noticed intermittent outages. This has to be solved. Not just in the short term, but in the medium and long term. Twitter has to be a service that just works. All websites suffer glitches and outages but the mean time to failure needs to be a lot higher and it should be cheap and cost effective to scale.

TwitApplications

There are a lot of services and applications that link into twitter. I consistently use tweetburner, tweetdeck and have looked at / considered a range of other services / applications. While the wiki page can point someone in the right direction. This needs to be integrated better into twitter itself

Facebook really took off and removed bebo and myspace as competitors, in my opinion the day it introduced facebook applications.

It should be a different process from facebook as facebook applications are of a different breed and different target market. Twitter simply needs to make it easier for applications to integrate in to solve two problems

  1. Easy launchpad to add them in and use them
  2. Remove the need to provide the twitter username/password in other websites. I currently have to do this with tweetburner to post directly which makes me very uncomfortable.

Accessibility

I am not talking about makes it easier for people with disabilities to access the site. I am talking about people who are not technically savvy or more importantly twitter savvy.

I joined twitter a while back and just felt a bit lost. There was no guidance as to what a tweet was, what it meant to be a follower or what it meant for people to follow you.

It took an article on a magazine explaining it to make it easier for me to understand and re-boot my twitter life.

Help & Support are good and useful but it should not be necessary if the help and support is present throughout the site. Facebook does this well and makes it easy to learn and do new things. It does not need to be idiot proof but it does need to have just enough information for a newbie to get started.

There are numerous blogs, articles and websites that cover this information but that means that someone has to spend enough effort getting out there and finding out.

This can be difficult when you don’t know what you are searching for as well.

Functional Integrations

There are several integrations that would be useful. There are websites that do some of these things but it would be useful to have them integrated within the site. Examples include:

  • Easy way to see the last tweet of all the people you are following / your followers
  • Popularity of the people you are following / your followers
  • Group people, so that you can follow people who blog about different things but read them together

Conclusion

From my perspective, this is of course a starting point, the tip of the iceberg. Twitter is involved in a lot of new things but without the soft aspect, I think it is making its life harder than it has to be to get the masses.

Making Twitter Faster

From my perspective, Twitter has a really really interesting technical problem to solve. How to store and retrieve a large amount of data really really quickly.

I am making some assumptions based on how I see twitter working. I have little information about how it is architected apart from some posts that suggests that it is running ruby on rails with MySQL?

Twitter is in the rare category where there is a very large number of data being added. There should be no updates (except to user information but there should be relatively very small amount of that). There is no need for transactionality. If I guess right, it should be a large amount of inserts and selects.

While a relational database is probably the only viable choice for the time being, I think that twitter can scale and perform better if all the extra bits of a relational database system was removed.

I love challenges like this. Technical ones are easier 😉

If I didn’t have a lifetime job, I would prototype this in a bit more depth. Garry pointed me in the direction of Hadoop. Having had a quick look at it, it can take care of the infrastructure, clustering and massive horizontal scaling requirements.

Now for the data layer on top. How to store and retrieve the data. HBase is probably a good option but doing it manually should be fairly straightforward too.

From my limited understanding of twitter, there are two key pieces of functionality, the timelines and search.

The timelines can be solved by storing each tweet as a file within a directory structure. My tweets would go into

/w/o/r/d/s/o/n/s/a/n/d/

The filename would be -

For the public timeline, you just have a similar folder structure, but with the timestamp, for example, the timestamp 1236158897 would go into the following structure as a symlink

/1/2/3/6/1/5/8/8/9/7/

For search, pick up each word in the tweet and pop the tweet as a symlink into that folder. You could have a folder per word or follow the structure above.

/t/w/i/t/t/e/r/- OR

twitter/-

You would then have an application running on top with a distributed cache with an API to ease access into the data easier than direct file access. Running on Linux, the kernel will take care of the large part of the automatic caching and buffering as long as there is enough RAM on the box.

This can in theory be done without Hadoop in between and separating the directory structures across multiple servers but that can have complications of its own, especially with adding and removing boxes for scalability.

You are also likely to run into issues with the number of files / sub-directories limits but they can be solved by ‘archiving’ – multiple options for that too…

Thinking about this problem brought me back to the good old days of working on the search mechanism within megabus.com. We needed the site to deal with a large number of searches on limited hardware when the project was still classified as a pilot.

With some hard work and experimentation, we were able to reduce the search time to a tenth of the original time.

I’ll admit that I don’t know the details or the intricacies of the requirements that twitter has. I have probably over-simplified the problem but it was still fun to think about. If you can think of problems with this – let me know; I wanna turn them into opportunities 😉

Work Life (Im)Balance

Started twittering this week although I’ve had an account on there for about a year. Huge, very interesting community on there of people from all walks of life. Whole communities, sub-communities, tools, utilities – all based on 140 character messages. It really is very impressive.

As a twitterer, there is the question I ask myself as to whether i tweet my personal life or my professional life. Right now, with half a dozen followers, it really doesnt matter. But in the longer term, it would be my goal to increase that number substantially. After all, it is all about who you know, not what you know…

I am already following Levar Burton (Geordi La Forge), Brent Spiner (Data) and Wil Wheaton (Wesley Crusher) – all from Star Trek The Next Generation, Richard Branson of Virgin and Jeff Atwood of Coding Horror and Stack Overflow.

I’ve often thought about separating my personal life from my professional life in terms of online presence. Clearly Star Trek above is a personal interest whereas the other ones are professional interests. But then, my professional interests are classified as such only because I turned personal interests into a career.

If I could make my interest in Star Trek a part of my career, I will…

I do wonder how it would affect my relationship with my followers though, in the long term. I am always going to tweet about things that are of real interest to only a percentage of the followers. This percentage is going to be lower than for other people. This stems partly from my wide range of interests and diverse group of friends / associates.

As an organisation, that works in a fairly diverse range of sectors, this identity crisis is experience not just by me, but as a whole within the organisation. One of the challenges we face is expressing the core values that are fundamental to the way we do work regardless of what that work is.

I believe, at the end of the day, that is the most important quality and our biggest USP…

Hello world!

Jeff Atwood asks in his blog:

“Until you’ve..

  • Written a blog entry about X
  • Posted Flickr photos of X
  • Uploaded a video of X to YouTube
  • Typed a Twitter message about X

.. did X really happen at all?

Kraya was a tiny company six months back with around 10 members of staff. We are now 20 strong and classed as a small company but as the artist formerly known as Prince once said

“I may be small, but so is dynamite”

Kraya has been instrumental to the dramatic success of megabus.com and not to mention the continuing existence of the Edinburgh Fringe Festival.

But then, as Jeff asks, since it hasn’t been blogged yet, did it really happen? While there are some pictures that could be published, there are no videos.

Besides, I’ve never been a big fan of Flickr or YouTube. Not that there is anything wrong with them but I use Facebook and that sates my desires for pictures and videos – for good or bad!

I never got into twitter either – the status updates within Facebook keeps me happy enough. Just as with Flickr, I don’t want to have to think about yet another website / system to update.

Now what I could do is add in the Facebook Apps for Flickr, YouTube & Twitter and this Blog into facebook and control my entire online presence from there. But then, do I really want “someone else” (Facebook) having so much control over my online presence.

Taking one step at a time, blog first, ask questions later…

WordPress Themes