Wiki + RSS + Email ecosystem

David Berlind, in recent ZDNet blog - RSS: The new intranet protocol? comments:

"With RSS as both the notification mechanism and the content subscription mechanism, you basically have a single technology that takes e-mail, e-mail attachments, and far too many round-trips (of email, to fully facilitate the collaboration) completely out of the equation. "

In essence, his main contention is that it is mainly email vendors (IBM, Microsoft etc.) who have kept the collaboration problem "alive" and not really providing better systems upfront in lieu of email technology. And now, they ought to take technologies like RSS and Wiki systems seriously and provide solutions that may replace email.

I have held to an opinion that Blogs are NOT a replacement for email. But here is a different take: RSS feed as notification of changing content in wiki, as opposed to carrying content. Is this feasible? Are people already using RSS this way? I am not sure. You normally "live" in email system, and go to read blogs in your RSS client once in a while only i.e. you will miss notifications due to faster pace of collaboration. Indeed many people wish to get emails of changes in wiki systems rather than use RSS for notifications.

So what is possibility that RSS clients take over outlook, and you start living more in RSS client? Not so soon, I would say.

There is standard "good-enough" notification mechanism: Those balloons which pop-up and disappear automatically in non-intrusive manner. Microsoft supports them in most of its products. Even thunderbird has them. May be someone has RSS-to-notifier tool out there. The only problem I find with them is their transient nature: If you are not in your seat, you are likely to miss it. But this can be augmented easily; all we want is a mechanism to show missed alerts/notifications when we return to seat.

So with multiple means of notification (i.e. emails, RSS and the pop-ups), I am not sure which are right means. All we know is that they should get our attention quickly. They also should integrate with other tools - such as postponing a task, creating an event and so on. Ideally, I would like a completely independent notification system that integrates with mobile phones, email systems and so on; it adjusts the visibility appropriately; understands expiry aspects of notifications well, and keeps tab on long as well as short term notifications. Perhaps some innovation is supposed to happen there. Yes, RSS and notifications are intricately connected; but pure RSS is not a solution for notifications-related issues I listed here.

Coming back to wiki vs. email vs. RSS problem, I believe that wikis will integrate with email in better manner. We once tried a project where email content (of a shared imap folder) was itself available as wiki, and changed content was simply posted as new email. That way, you have same content as both wiki and email; solving a major integration problem. But the project was not fully explored.

Another approach is to "webify" email i.e. have a set of tools/technologies that will transform email consumption approach to use browser (as opposed to outlook etc.). That means, every email should be a web resource. And that will enable us to create shared areas easily (to share the emails, its documents etc.), and more importantly, start shifting to wiki based collaboration in incremental and integrated manner. So over the time, everyone will find it easier to simply use wiki approach to write documents and create presentations rather than using Microsoft products for the same. (It also lessens the IT support required to support single web stack in the organization.) Zimbra pioneered this type of enterprise stack already though it is still emai-only.

One most important aspect that still matters: Emails should be available offline - on laptops, and "with you always". This is conceptually as well as technologically strong argument (i.e. less dependencies; you always have access to your email even when internet connection doesn't work etc.). But when we also want web technology to creep in somehow instead of FAT outlook. The only architecture is to have offline web server, and synchronization mechanism that treats all content uniformly. Some companies (webaroo comes to mind) are already pushing this platform (For saved search results and ready-to-use web-packs).

And of course, we know Google now has capability to offer enterprise email systems - although no offline means as yet. We can guess they have a plan.

So there is indeed a lot of excitement in this space, and email ecosystem is still most happening place for innovation.

ComputerWorld article: The Future of E-mail

The Future of E-mail explores various research related to email - from Microsoft R&D, IBM and HP, among several others - along with titbits such as how many work in email space in Microsoft R&D (which is, about 40). Here is quick summary of all projects:

Microsoft's efforts in Malware control
- "Combination of New technologies, plus economic and political pressures will solve the problem." Microsoft is primarily pushing Sender ID framework, whose advantage, it claims, is that it will work across the communication technologies (such as IM).
- "SmartScreen" filter uses statistical approach to identify spam. Phishing filter add-in for MSN Search toolbar. Other approaches - such as solving puzzles, micropayments also under consideration.
- "MailScope" which monitors email routes and identifies delays in email delivery.
- "SureMail" - approach to ensure emails are indeed received. A parallel architecture to email to post notifications (of having sent email) into a centralized table and ensuring that clients poll that table to know about pending emails.

Email as part of activity threads in workplace
- IBM's Activity Explorer is a collaboration tool that pulls together e-mail messages, synchronous communication such as instant messages, screen images, files, folders and to-do lists. A different UI has emails under explicitly defined activities of a business process.
- Microsoft Research has developed a way to combine e-mail, files, Web pages, calendar entries, to-do lists and other materials into one searchable archive. Called "Stuff I've Seen".

Mining corporate message archives
- HP's message mining to identify patterns of collaboration among people: Leadership roles, people who act as hubs and so on. For e.g. "Who are top 5 experts in topic XYZ?"
- Also a similar (but more subtler approaches) from Jon Kleinberg, a professor of computer science at Cornell University, who focuses on social networks; understanding precisely who or what factors are responsible for key influences on community.

Email has been my personal focus for last several years, and I can't agree more to a general impression that email is yet to come out of its teanage stage. Blogs serve a different communication paradigm, and while it will co-exist (and integrate - from workflow point of view), it won't replace email. But wikis are very different beasts; if done well, they have capability to create a dent in email. It all depends on how universal they will become; like how sendmail became universal some years ago.

We have some projects within my organization, and quite interesting too - it has elements of all three aspects discussed above. You would have to contact me through official channels, if you are interested.

Success of "Bhoomi" - eGovernance project of Karnataka state govt.

Slowly and steadily, e-Governance projects have started making a mark in India. From: Right click to e-asy governance- The Times of India article:

- It just takes 2 min. to get your land records. (Compared to endless queues earlier + bribes etc.)
- 4.06 crore farmers have used this service, paying about Rs. 61.94 crores for the service; roughly about Rs. 15/- per use. (This is probably for period of about 2000 to 2005.)
- Most important, the service is available via 203 kiosks all over the state.
- To ensure there is no unathorized tampering of records, officers have to use their fingerprints to get access.
- Linked online with banks and courts, all associated processes (such as taking loans, resolving litigations) have become much simpler and taking less time.
- Estimated Rs. 2500 crores savings from controlling losses due to tampering from Bangalore alone.

This successful implementation and goes a long way to give confidence in the eGovernance systems.

Even in Maharashtra, the system has been put into place.

Maturity from the governing class is still slow to come. In a recent local news, the corporaters of Pune Municipal Corporation demanded a return to manual system to collect the taxes because servers were not functioning properly for a month. They could have as well demanded to know why PMC is not able to maintain its servers in nice way. (Perhaps it is due to common problem of not having to find right people/companies who can provide such services in reliable manner.)

Whatever it is, we see strong winds of change, and it is bound to take everyone along with it ...

Cringely to local TV stations: Connect to your customers directly from local telcos ...

Here is something that can greatly benefit India.

Just some time back, my house had 100mbps fibre connectivity via a local ISP provider. However, the business model of this provider was not strong. And he had to take the fibre over the existing housing societies, running into problems with cable providers. Also, there was no easy solution to provide power at different points which needed bridging. Ultimately, that provider closed down.

What I had suggested to him at that time was to install a set of services involving audio/video from the free content available from internet, and to use local advertisements effectively. Essentially make the service sufficiently sticky. But they just couldn't do it. This big amount of bandwidth - connecting some 2-3000 houses - could have also been used in P2P manner. But then it would have required something else altogether.


Similar model is in essence the summary of Cringely's suggestions in (PBS | I, Cringely . June 8, 2006 - Local Heroes). He suggests that local TV companies can make use of this approach - by keeping their servers at each of the DSL/Cable provider's network room.

In contrast, centralized, web-based approach to serving video / broadband data to end-users is very costly. In India, you can today afford 256kbps at reasonable costs. Nothing more. So Cringely's model is a very interesting hope - provided our policy makers take note. (Rather, a hoping that they don't interfere, if such a model boots up.)

This is surely a model that can be kicked off with purely local relations (as Cringely points out). In India, we still don't have concepts of local TV stations. Local cable companies do show movies and (sometimes live) footage of local events. And the biggest benefitiary would be the education sector which enables eLearning on a wide scale. (Folks at Project Ekalavya, take note!).

Reliance already is already testing IPTV with its 100mbps network. (And I have seen it, and it is simply fabulous - you can access any archived TV program, choose any movie at any time, or listen to songs.) I am not sure of other players, but Cringely's model doesn't require centralized approach, it all depends on whether they can inflence BSNL (key player in DSL segment) and several other ISPs in India. Clearly an opportunity for a new business model - if someone has right contacts.

Enterprise deployment of Firefox ...

Biggest problem of using things like bookmarklets in enterprise is that you can't cleanly ensure everyone can see same thing. You require some decent means to control the configuration from one place.

The mozillazine article - Firefox Enterprise Notes has some best practices - though bit old (July 2005).

Hope that Flock, with its recent beta, can make it to enterprise!

India's answer to "Ripley's Believe it or not"

Zee's "Shabhash India" show debuted today with amazing believe-it-or-not type performances from individuals from different corners of the country.

First performer took in water from his nose and threw it like a jet, from ... corner of his eye! That this jet of water was so forceful was it was thrown to distance of about 11 feet. Think about the kind of cavity the person may be having, and the kind of control on his internal muscles.

The second person was having, visibly, the largest lung that I might have seen so far. This person took a truck tube, blew it to fill it completely .... and then blow so much as to burst it!! To be able to do this, his lungs should bear forces greater that what a truck can create when tyres burst!

Both the above people claimed that their capability came from Yoga/Meditation/Pranayama.

And the third one was a small kid talented with skating. In skating, one of the performance is to widen the legs and try to pass through a clearance as small as possible. Now this kind managed through of clearance of 6.5 inches... which was created by lining up some 33 cars side-by-side!!! That is at about 100 to 120 feet long!! Probably the height of this kid's head was about 5 inch - when in horizontal position - turned sidewise. Also, I don't know how he moved all that 120 feet in straight line - because he held his body rigidly throughout.

All in all, Kudos to Zee for having creating this interesting TV show, and having identified such interesting people from different places. The show is going to be there every day ...

Tagging email messages - Thunderbird plugin from tagthe.net!

Finally the tagging comes to email world. tagthe.net blog has a release candidate for the same.

Emails and web world are still separate. Ideally, we should have a standardized approach to view any emails as a web resource (i.e. URL), and then be able to tag it or use tonnes of innovative approaches out there...

BTW, tagthe.net service (which automatically creates tags) is also quite amazing... Unlike recommendations given from sites like delicious, I guess this service uses text analysis.

BarCampPune - Recoja tags specific content within web pages

Bringing tagging granularity to finer level so you can tag at paragraph or specific content, is an innovation that is likely to be very useful. What it does is to allow us to increase relevance of specific content, and not web page as a whole. This is greatly useful when you are consuming content in a aggregated manner (such as in news aggregators).

Shodhan Sheth and Anand Kishore are working on Recoja, which does precisely this thing. They presented their work on extensions to delicious tagging to facilitate this (via firefox plugin).

What Recoja does is that it allows you to tag a specific paragraph within a web, so that it is much more context specific. So when page is displayed (in firefox), it will show a small tag icon in green (cute, really!) and then when you move cursor there, it will highlight the tags put there.

There are some challenges. For e.g. the tags are external to content. So when content changes, their algorithms have to re-identify the content which was tagged. So the references which are created should be content based. (One algorithm: Choose the paragraph, checksum it and store it. Next time, try to identify this paragraphs. Use 2-3 trigger words to try to identify the checksum that needs to be search in current page.) But this algorithm requires a good amount of CPU power, as well as storage.

Now that we create tags at paragraph level, then this will basically allow us to add and share notes when collaborating - better than the way Web does. This is very much required for wiki systems!

(Questions asked: Can I use the same to tag audio/video content - specific segments?)

Good plugin!!

BarCampPune - Context based search project by Mukul Joshi (and other presentations)

Mukul Joshi presented an implementation of context-sensitive search engine that was implemented by some project students, and got a first prize in annual competition held in IISc. (Sorry, don't have reference here.)

Context search is when search engine gives you options when you search. For e.g. if you say Apple, is it Apple computer or Apple fruit?

He gave a good overview, and explained all the concepts; in particular use of Context Net database (again, sorry no reference as yet) that has some 50000+ concepts. More importantly, how can personalization be integrated so that user doesn't have to specifically select context. He also neatly summed up various problems in search space still to be attacked.

Interestingly, Webaroo also gave a presentation. They have something known as 'webpacks' which are a set of collected information about a particular topic such as City. (These can be uploaded to mobile later.) So that is basically a context, right? So what they do is to automatically identify good resources and pack them for you to download them in mobile (so they are available offline for search and browse).

Atul Kulkarni gave a talk on Naive Bayesian classification, and gave resources / toolkits etc.

BarCampPune - Context based search project by Mukul Joshi (and others)

Mukul Joshi presented an implementation of context-sensitive search engine that was implemented by some project students, and got a first prize in annual competition held in IISc. (Sorry, don't have reference here.)

Context search is when search engine gives you options when you search. For e.g. if you say Apple, is it Apple computer or Apple fruit?

He gave a good overview, and explained all the concepts; in particular use of Context Net database (again, sorry no reference as yet) that has some 50000+ concepts. More importantly, how can personalization be integrated so that user doesn't have to specifically select context. He also neatly summed up various problems in search space still to be attacked.

Interestingly, Webaroo also gave a presentation. They have something known as 'webpacks' which are a set of collected information about a particular topic such as City. (These can be uploaded to mobile later.) So that is basically a context, right? So what they do is to automatically identify good resources and pack them for you to download them in mobile (so they are available offline for search and browse).

Atul Kulkarni gave a talk on Naive Bayesian classification, and gave resources / toolkits etc.

BarCampPune - Adaptive Learning by Parag Shah ...

Parag Shah, an avid researcher on learning methodologies, put up a good list of productivity tools in this space at Adaptive Learning.

For e.g. ePortfolios as one of the tools. Another example: Virtual worlds, such as SecondLife, are going to be one of the key enablers. (It seems Microsoft has already put a lot of stuff there for learning.) Some scenes from second life put on the site.

Some good practices out there to check out.

BarCampPune - Carpools for India!

Finally some interesting problems getting solved in India. In this case, carpooling - for key Indian cities: Sidharth Sharma, who lives in Pune, has setup http://www.patang.org/projects/ajaxcarpool/carpool.php.

It is still not a service, but currently a sourceforge project. Needless to say, integrates with Google Maps!

And since we don't have good road maps to go with google maps in India, hopefully this effort will become a vehicle to augment google maps with road networks in a collaborative manner.

BarCampPune - Indecca uses Erlang for gaming infrastructure

Today at BarCampPune, Rakesh Raju from Indecca, a gaming infrastructure company gave a good presentation on their technology challenges in supporting gaming infrastructure.

Requiring scaling to lakhs of users, they are considering Erlang for their infrastructure - which has been used for telecom industry for scalability, concurrency etc.

Seems like some good thinking going on there.

Intel's (new?) site layout and feedback usability ...

I visited Intel website recently and was in for a pleasant surprise! Intel has given up on those dark blue colors, and seems to be using best of the web design techniques.

In particular, I was bit intrigued by the way they get feedback. A small window appears (not a popup; but those javascript effects - such as this. But it only asks if you are interested in giving feedback after reading the article, and if you click on yes, places a special icon at bottom right corner of the page.

It is a different story that in firefox 1.5.0.4 (i.e. most recent update), I couldn't access that icon. The firefox scrollbar would roll back just above that icon everytime I try to drag it to try to access that icon. Obviously some gotcha - when it is placed relative to frame.

The article I was reading was about the kind of challenges in Intel's mobility strategy (Building Intel's mobile tomorrow - an interview with Kevin Kahn - Director of Communication Technology Lab at Intel. I still retain interest in Intel's Mobilized Software Initiative (though it has kind of gone out of buzz) - with its objective of truly mobile computing (i.e. even with limited internet connectivity, using power judiciously, designing for multiple form factors and so on.)

Insight into Hotmail's infrastructure

A Conversation with Phil Smoot - An engineer at Hotmail discusses the challenges of keeping one of the Web’s largest and oldest Internet services running 24/7. A good overview of their challenges and methodologies of large scale system.

Summary:

• 10000 servers, 100 admins (across the world)
• Primarily Microsoft shop. (Visual studio, SQLServer etc.)
• billions of email messages (mostly spam?!!) per day

Key challenges:

• Dealing with abuse and spam
• Shipping features every 3 and 6 months
• How to release complex changes over a set of multiple releases
• Best practices
• Fully automated rollouts
• Distribute new version to first 10 servers, and then 100 and then 1000 and so on. Ensure multiple versions can co-exist
• Load balancing across the site. (transactions + user capacity)
• Focus on specs of changes, so as to be able to predict the effects and measure them. Be able to have right instrumentation for measurements.
• Predict cost of change. Even a penny counts (over millions of users.)
• Be able to operationally you can isolate a broken or failing hardware component or server or service.

Screencasting as Advertisement medium ...

A microsoft video ad screams that "you can be king of web" through a video that shows a man picking up a crown from a drawer and wearing it (in his cubicle.)

Why can't they instead directly screencast the actual product usage and capabilities? Considering that I was viewing the page of a highly technical site, the devels and managers might as well get some really useful information and "get involved" with the advert.

May be some other companies can try that!

-Vinod

Amazon CTO Werner Vogel's views on SOA standards ...

A great inerview in ACM Queue: A Conversation with Werner Vogels - Amazon's CTO explains what's behind its growth from online bookstore to e-commerce juggernaut.:

JG: "How many of the current buzzwords, such as SOA, WSDL, SOAP, WS-security, are relevant to you?"

WV: "I would like to distinguish three categories of interfaces here. The first category is the services that make up the Amazon platform. There we use interface specifications such as WSDL, but we use optimized transport and marshalling technology to ensure efficient use of CPU and network resources.

The second category is the interface with our retail partners, which has strict descriptions for XML feed processing, service interfaces, etc., and where we leverage as many standard technologies as possible.

The third category is our public Amazon Web Services, which builds on the platform services and provides REST-like as well as SOAP interfaces. If we look at how developers use these interfaces, in general the REST version is used by small libraries in Perl or PHP as part of a LAMP stack, and the SOAP calls are mainly done by applications that have been built on Java or .NET platforms by consuming our WSDL files and generating proxy objects.

Do we see that customers who develop applications using AWS care about REST or SOAP? Absolutely not! A small group of REST evangelists continue to use the Amazon Web Services numbers to drive that distinction, but we find that developers really just want to build their applications using the easiest toolkit they can find. They are not interested in what goes on the wire or how request URLs get constructed; they just want to build their applications."

So REST vs. SOAP/WebServices debate continues. "Easiest toolkit" is of course what all developers are looking for - ones that hide all the intricacies and dependencies among multiple-vendor stacks out there. But would REST win (because of least dependencies, simplicity)? Would Web Services win (generally elaborate architecture, more vendor activity)?

Raj National Travels ...

Raj national travels now offers a complete internet based booking (http://www.rajnationalexpress.in/) from any point to any point (on the routes they have services).

But one of the foolish things they seem to be doing is to imitate the Airline business to obsene levels. For e.g. you have to "check in the luggage", "report 30 min prior to the start time" etc. The whole point of traveling by bus - especially between cities such as Mumbai and Pune (distance of about 2 hours) is that you save a lot of time by catching the bus just 5 min before scheduled departure.

And another foolish thing: Because they run off the internet, they don't have mechanism to admit last-minute passengers. Once we were right at there when bus was about to leave (from the last pick-up point in the city), and even though there were sufficient no. of unoccupied seats, the in-charge basically said they can't any more passengers. No - not even "in-house" TC as you have in trains! Isn't that foolish? Basically it shows how the software systems are often built without understanding of real world properly...

The level of sincerity among staff is however quite high. When the service was delayed once, the person in-charge gave factual answers, and also gave appropriate instructions to not to stop for tea break - since we had to catch flight immediately.

In any case, the rates are no less than other buses. The main convenience is that you can book with your credit card (and beware - you may not be able to cancel; the cancellation charges are more than ticket charges - another fooling thing!).

Hope these people get software things right. And most importantly, they should give some respect to most basic aspects of life, and not take us customers for granted.

Titbits from Frank Addante's FounderBlog: From a 29-year-old college drop-out

Frank has written a nice blog piece on his experiences with startup (http://strongmail.com/). Strongmail is in very interesting space - which you must track if you are in email space.

Some really interesting points made there:

About sensing capabilities in people ...

"This was one of the largest, most demanding IT infrastructures ever built, and Tim conquered it; a guy, who less than 2 years prior, was hired to be our recruiter. This proved to me that smart, motivated people, with a passion for learning, can accomplish anything if given the right opportunity..."

About how SPAM, which in traditional way of looking, was going to be bad for their business, but then became opportunity for the company ... since everyone wanted even better control over email.

"At first, I thought this was going to destroy our business – email was no longer trustworthy. Then, the opportunity became clear. With all of the changes came new laws, new protocols, new industry standards and a whole lot of complexity. These things would require every company in the world to adapt their existing email infrastructure to deal with the changes, and keep up with the rapidly evolving requirements. It turned out that this drove even more demand for StrongMail. This became the immediate compelling event."

On cultural differences between LA an Silicon Valley ...

"We did not fully appreciate or understand the different nuances in the work culture between Los Angeles and Silicon Valley. With tech companies in Los Angeles, people show up to work in jeans, are laid back, work hard, have diverse work experience (beyond software) and simply focus on results. In Silicon Valley, people are much more competitive, have more focus/experience in software backgrounds and often focus more attention on the methods first, then the results."

On importance of gut-feeling; when trying to "go-by-book" didn't work ...

"I promised myself that I would always trust my gut and my instincts –right or wrong- and move forward. I needed to feel confident in my decisions. That’s exactly what I did and almost overnight, the entire company was transformed – we started growing quickly, exceeding our targets and we have been seeing extremely positive results ever since."

If you are on enterpreneureal track, don't miss it!

PCQuest nominates Best IT Implmentation Awards for Indian projects ...

Railways gets first prize for their operational system designed by CRIS, among 152 projects nominated from 25 industry verticals.

The data seems to say that 1. IT is high priority for Indian businesses 2. The projects are largely successful. (Though we can't really conclude the second point, because there is no data about how many projects are delayed or have not been executed properly.)

An interesting study of its kind by PCQuest, all the laurels for it.