Your data is your memory - we need to make it searchable

I confess, I’m a digital hoarder. I have back-up drives, copies of hard disks, I keep hard disks from old laptops and desktops, I have old SD cards from cameras, old USB sticks and even old floppy disks. I don’t want to let go: I feel that all those documents and files are a part of me somehow.

My memory is no longer solely in my brain, or written down on paper – it is almost entirely digital.

Add to this that I am now buying digital products through Amazon, iTunes, Sony and more, I realise that the scope and size of my digital identity is only going to grow. (Please see a prior post on what a digital identity is here.)

My underlying idea here is simple: humans mostly think in terms of relationships between data points, we don’t think in terms of structured database tables.  We remember things in terms of relationships, and often uncovering one memory triggers another whole set of associations that can lead you to the answer you want.

As an example, many people I know like to use archived emails to track down information because they remember a date or a person they associate with a piece of information (yes, I keep email archives going back 10 years just to do this.)

Use in the workplace

Let’s take a workplace example that I could use in my consulting practice.

I get a call from a client asking me to reissue several reports as well as tracking down the original source documents for assumptions.

I have a lot of clients, and despite best intentions, some files remain on emails, some in saved documents on my computer, some on the cloud, and some in notebooks.

What I’d love to do is to let the system know the name of a person, a year and month and the name of a project.

The system would then track down any documents with those parameters, then give me a search outcome. I could then choose a couple which were most relevant, then the system could go and complete the search.

What I would love to see is:

•    A list of folders with relevant files.
• The documents in those files.
• All email threads associated with the  project – starting with direct communication with that person, but also then expanding out to other people associated with the project.
• Any calendar entries associated with that project. It would be even better if I could click on a calendar event and then have documents created before and immediately after that calendar event show up in a list.
• Contact details for any person involved with that project.
• Could I even see when I was making calls or receiving calls from people involved with that project.

 Alright, this might be a bit much, but you get the idea.

Use on a personal level

You are at a dinner party with a friend and they are about to travel overseas to a country or city you have gone to before and they are asking for recommendations on where to stay, things to see, places to go, and places to eat.

Okay, this might seem a trite example, but wouldn’t it be cool to do a search based on August 2010 (when you went on that trip) and check your photos, check your itinerary on Google Maps (or equivalent), check your credit card or bank records for the name of the restaurant or other place, etc.

As a second example, let’s say you were really moved by a painting you saw when you visited an art gallery years ago. This isn’t as stupid an example as you might think. Let’s say you are in your 60’s,  you have terminal cancer and you want to relive some of the really positive experiences of your life (this example is near and dear to me at the moment as I just lost my mother to aggressive cancer at with little warning. I would have loved to have been able to help her remember some of her fondest experiences at the end.)

You would like to see the painting again, and want to find out the name of the painting and where it is now. Wouldn’t it be nice to see what shows were on display at the art gallery at the time, but wait… you only remember that it was in 2007 and you were on holiday in a certain city. What if you could easily track down where you visited in that city, then the few galleries in that area, then see what was on display, find the painting in question and then find out where it is today.

A third personal example would be trying to track down a song you really liked in 2005. Right now you can search for music released in 2005, you can search for the top 100 hits every week in 2005, but what we can’t do is search for a song we used to play all the time in Spring 2005, which may well be a song from the 1970’s.

I don’t know about you, but as my digital music collection grows, I am finding it harder and harder to make sense of my music collection. An album I liked 10 years ago may not be one I want to listen to now. On the other hand I may be nostalgic for music I liked 10 years ago.

We tend to remember music from times in our lives and relationships. It could have been a friend who introduced you to some classic jazz from the 1930’s, it could have been a particularly great Summer vacation, it could have been a song that helped you get through a painful breakup.

As a fourth personal example, this concept could be of use in budgeting, and other aspects of our lives.

These days we make most of our purchases using electronic banking. Accounting software already allows for transactions to be directly loaded from bank statements and credit card statements. On a personal level, some people may want to keep track of the same. This could also be taken to the next level.

What if by clicking on a transaction at a supermarket you could also get the itemised docket. You could check what you were buying, the price of it, how much of it you bought and even its nutritional value and calories.

This might sound over the top, but look at it another way, if you have ever met anyone trying to keep a food diary to lose weight, or to kick substance abuse (whether it’s alcohol, caffeine or sugar), you will know how hard it is. Not only do we all hate manually entering data, but one of the strongest characteristics of human beings is our ability to bullshit ourselves about what we are actually doing. You can’t improve without measurement.

Having this kind of data available for yourself or your doctor, or your accountant could be quite valuable.

We are reaching a threshold

Our lives are becoming more and more complicated and we are relying more and more on our digital memory, and I think it will become harder and harder for us to have that digital memory erased every time we delete a hard drive, or change over a phone or phone provider. That data should be ours, not that only that of the company that provided the service.

It’s only going to become more complicated: in the very near future we have the internet of things arriving where we will connect our household, workplaces and cars to the internet. Also we are entering the world of 3D printing for physical objects ranging from items we use daily, through building construction to now research is proving up printing replacement organs for our bodies, and even printing food (the US army is looking at this.)

Just as I write this blog, news has broken that Delaware has just passed laws allowing families to inherit digital assets. This is a great example of what is coming. Now family members are legally entitled to access all digital assets, including Facebook, Twitter, and I presume iTunes and Amazon, despite what the terms and conditions for those web sites are.

Yes, there are practical limits to this today in terms of cost, but given projected changes in the cost and capacity of digital storage and processor speeds over the next 5 to 10 years, this may all be closer than we think.

It’s also not like this is a totally new concept

Google has tried to expand its search capabilities from websites by including books, movies and music. They previously sold their search capabilities for internal use at companies where a rack mounted unit could classify all files in the company servers and make them searchable.

There are also a lot of efforts to make images searchable too (e.g. using a meta search like, Rembrandt, lady, small child, tree in background, or as another example, ocean view, Gold Coast, 1950’s).

Some of this technology is already coming of age through intelligence analysis, forensic accounting and legal discovery processes where the automation of all this can occur because the time, effort and costs of this are considered worth the benefits.

For example, when doing an analysis of procurement fraud in a company it is possible to mine (where legal) facebook, Google+ and Linkedin contacts for an individual and see their relationships with suppliers. This can then be used to automatically sieve through the terabytes of data retained by a company to find the key documents to be used. Many instances of fraud aren’t terribly well disguised, and if someone is getting a clear benefit from a supplier in the form of kickbacks there is a pretty good chance that they will become friends online, if not in person.

The intelligence community is already using these methodologies to try and monitor for terrorism, organised crime and other forms of crime. Think of the NSA revelations that Snowden exposed. This is where the whole metadata question comes in: they want to keep record of what numbers you received calls from, and who you called, at what time, and the same for emails. They don’t necessarily keep the content of those calls or emails, but they can see the relationships. They have another layer where the system monitors phone calls and emails for keywords, and then starts mapping relationships between people when trigger events occur.

Lawyers on complex lawsuits are starting to trial the use of intelligent searching during the discovery phase of the lawsuit. What this means in practice is that you get given several terabytes of scanned documents and PDF’s that have been turned into images so the text isn’t searchable. Instead of paying a large team of junior people to spend months combing the files and creating a searchable database, the emphasis is now on making the files searchable through the use of optical character recognition, then running rules based searches to look for keywords, frequency of keywords, authors of documents, dates of documents, recipients of documents, etc. That is, the system maps the documents and you just perform a Google type search on the documents to pull up the relevant ones, and see relationships with other documents.

These systems exist, and they are expensive to run as they require an expert to use, and as you’d expect the real cost comes with finding the data, then formatting and reformatting the data into a form you can use (called ‘data munging’ or ‘data wrangling’), then linking the data correctly into the search program.

What I am talking about is bringing this to a personal level or a corporate level where you have greater control of providing permissions to your own private data sets.

I want the future now

There has to be a way to start stitching all this together as we go forward as part of our digital identity.

• We need standards on what data we own.
• We need to be able to retain our own data, and have strict permissions and access based on the who is trying to access it, and setting in stone privacy laws and rules. This gets weirdly complicated where your data and digital assets may be distributed among servers around the world.
• We need documents, files and digital data to have standards on tagging/metadata so that we don’t have to automate classification.
• Even keeping track of things like when I played a piece of music, or watched a movie, or read a book may be something we can keep track of. There will be a growing need for standards on this.

I wish I could live in the future, and I hope someone out there is working to make this happen. (I’d love to hear any examples of work in progress and ideas on other ways this could work.)

Auto-nagging and other ideas for the future of Google Glass

Since Google glasses came out I have been thinking a lot about the future of computing and apps. I think we are not far off turning science fiction technology into everyday reality.

The topic that got me started was thinking about friends with vision problems or impairments. Red Green colour blindness runs in my family and I was thinking – wouldn’t it be great if the glasses could identify a red light and flash “STOP” in your vision.

Of course I am not talking about Google Glasses as they are now, but a future device which is a powerful computer as well.

So – how about the following list of ideas.

Colour matcher for the colour blind (already an app for smartphones). Never buy a purple shirt again when you thought it was brown.

Facial recognition for the blind – hear their name. “Jessica has entered the room.” or “David is looking at you.”, or “a stranger is staring at you”.

Facial recognition for networking – see somebody and you can be reminded who they are, pull up notes from recent conversations, etc.

When travelling, see a translation of what you are hearing in a foreign language in English in your glasses.

See subtitles on a foreign language movie through fast translation.

Read a foreign language newspaper in English.

Pick out a face in a crowd – good for not only police looking for a suspect, but also at any time you are waiting for, or looking for someone – and can use the higher sensitivity of your camera than your eyes.

Lip reading – great for the deaf, but could also assist with translation too (yes, and of course spying too).

Look at an object in the shop and pull up comparative models from other brands, and pricing. “this model is available for $10 cheap within 500m of here”

Do I have any eggs left? Automatically see an image of what was in your fridge the last time you opened it.

Is my garage door shut? I can’t be the only person out there who leaves home in a hurry and then can’t remember if I shut the garage door. How about your glasses take an image and reminds you. Or it could be linked to a camera of the garage door.

Looking for directions. You are walking down the street and you see an arrow pointing you in the right direction. Or even better, it highlights your destination.

Visual groupon – walk down the street and see the deals of the day flashing in front of the shopfront.

Find your cat or dog. Not only put a little GPS tracker on your furry companion, but when the little bugger gets lost, you get visual directions to their location. You can see a flashing beacon or suchlike when you look in the direction that they are in.

Keep an eye on your kids or pets from another room.

Create a food diary that is 100% honest. Everytime you start chewing sensors pick up the vibrations and take a photo of what you eat. Never say “I forgot about the Mars bar” again.

Auto-nagging – my favourite idea and a few years off. Everytime you look at high calorie food it asks “Do you really need this today? Your calorie intake for the day is already too high. Think about your health.”

Keep track of your household budget – just look at your shopping docket (receipt) and the optical character recognition will add things up. Link it to your expenses and banking apps and you will get an alert in real time.

Recognising food for the blind – to a blind person cans and packages all tend to feel the same. How about optical character recognition reads out what food it is. This could be based on bar codes to start. They could use it while shopping to try new things, as well as to recognise food when they are home. Foodmakers could have their cooking/preparation instructions linked to a QRcode or something on the label.

Recipe suggester – don’t know what to cook tonight? Don’t want to go down the shops? How about you just have a look in the fridge and cupboard and a series of recipes come up based on what you already have. You could choose cuisine types “Indian please’, or how long it takes to cook, or recipes by famous chefs.

Reporting accidents – upload images and video to the government, police, emergency services, insurance provider, etc. It tags information with location, time stamp, and even licence plates of other vehicles.

Clothes shopping – See some clothes you like, then put them on your avatar to see how you’d look in them.

Buying new glasses or sunglasses – again, try them on your avatar and see how they fit your face. The makers would put the exact dimensions online so you’d see exactly how they fit.

For the nature lovers – how about recognition of birds, spiders, bugs, moths, etc. Not only can you find out which species it is, but you can record the date, time and location which will help all the ‘twitchers’ out there and gather data for science.

Virtual reality cenotaphs – when you visit places that you associate with the deceased, see photos or videos of them there with you.

Visible invisible friends – for the kids in all of us. Have an invisible friend that interacts in real time with objects and people around you.

Night vision – need I say more. This would be awesome, and breakthroughs are starting to be made.

Black and white vision – this may not be an obvious one, but for visual artist seeing the value of a colour painting is important. Abstract artists could more easily create figure-ground effects. You could also do paintings with imposed colour blindness – which might seem stupid, but some artists seem fresh or unique because they see colours differently.

Guided tour around the Louvre – access the Louvre App when you are there and it will pull up details about the objects you are looking at. If you are short on time, it will provide guided tours by theme that lead you around the gallery.

Virtual cosplay (costume play)– lighten up your day. Get your Google Glasses equivalent to turn all your co-workers into cartoon characters. Your boss annoying you? Well turn him into the pointy-hair boss from Dilbert. Turn your team leader into a superhero or villain. Put Angeline Jolie’s face over the top of less attractive co-worker to make your life just that little bit visually pleasing. All done with facial recognition technology.

Coping with dementia – facial recognition of loved ones, a return home when lost navigation tool, set a geographic boundary that will trigger an alarm if they cross it and notify you where they are.

Tinder on steroids – imagine eHarmony or RSVP, or tinder for those more active types, using facial recognition. Highlight the faces in a crowd who might be compatible. Eliminate them by double blinking. Nothing new here. Fifteen years ago the Japanese had pagers with proximity sensors and fetishes loaded so people who met each other in public would know there was somebody else up for the same mischief nearby. This is just the updated version.

Trouble getting it up? – how about you overlay an attractive person’s face on your partner. Put your favourite movie actor or actresses face in place of theirs and performance is guaranteed. This could be the new way to spice up your sex life – take roleplaying to the next level.

Visual instructions – want to change the sparkplug on your mower. A visual overlay will guide you. Don’t know where the fuse box is in your car – let a visual overlay show you.

There are so many more ideas out there I know. This is just a selection of ideas that have come to me, and I am sure the rest of you have millions more.

I can’t wait for the future to happen.

Your GPS thinks you are stupid

I have a confession to make, I am not longer enamoured with my GPS. 

 

It started off as a passionate relationship where I used it every chance I could to revel in amazement at its technological mastery, then I went through a phase of trying different voices (male, female, Australian, US, Canadian or English), and even trying the different languages so I could learn some basics of French and German as I drove to work every day.

However, like all former favourite toys, my GPS has been sitting in a drawer in the house for a year or so now.

When I think about what tore our relationship apart it was the way my GPS always considered me to be an idiot. Every time I started it up it gave me directions on how to get out of my own street.

Well, I have a fix. Imagine the following dialogue with your GPS instead.

Good morning, please state your destination.

54 Mallabar Road on the Gold Coast please.

Okay, do you know how to get to the M1 southbound to the Gold Coast.

Yes

Alright, please head there and when we get close to exit 63 I will reactivate to take you from there

Imagine your GPS giving you the following instructions.

Do you know the big Westfield at Carindale?

Yes

Well go past it on Creek road southbound till you can see the McDonalds sign. Turn right at the lights just before McDonalds, keep going till you get to the roundabout and then turn left.

Imagine your GPS remembers your friends place or places you regularly go to.

Do you remember how to get to the Indian restaurant you went to last Saturday?

Yes.

Please head there and I will reactivate to give you precise directions when you are close.

Or, how about the following.

Do you know how to get to Kedron?

No.

In that case, do you know how to get to Enoggera.

Yes.

Please head towards Enoggera and I will give you precise directions from there.

Wouldn't that be fantastic. Let's face it we know our way around the city, even unfamiliar parts of it to some extent or another.

We can navigate by street names, major intersections, suburbs, shopping centres, petrol (gas) stations, places we go to regularly, sports grounds, golf courses, etc.

An obvious problem with the ideas is that these places change over time. Well, yes, and that also raises the question of who keeps the data up to date. So lets start with suburb names, major road names, major shopping centres and major intersections. Those don't change too often.

Ultimately, it would be nice to have this all through your smartphone, possibly only a voice based system with visual maps only if and when you need it.

This makes sense to me as it is how I and most people I know navigate. Once you have familiarity with an area, you look up a map to see if there are any landmarks or waypoints near your destination that you are familiar with and then map out your route from there. What we never do is work out how to go the whole way.

So, here's to a revolution in GPS to make it match us humans.

Some startup ideas I’d love to see

 Here are some business ideas I'd love to see become real. 

An online junkmail account

 Imagine controlling the advertising you receive through your own account.

You can select the products you want to know more about, and local supplier’s brochures are automatically sourced and loaded onto your account.

Looking for a new car – end of financial year specials coming soon.

Looking for a new sofa – here are the latest deals.

Looking for a new computer – here are the hottest specials.

You name it, for work or home, I’d love to see it.

This would provide a direct marketing opportunity for companies, and would allow you as a consumer to control the deluge of catalogues you receive. It would also help save all those countless trees sacrificed to advertising catalogues for things that people aren’t interested in.

E-book libraries for companies

I love my Kindle and have started buying a lot of reference books on Amazon now.

Imagine being able to buy a few copies of a book, and being able to lend it out internally in your company. For example, only one person at a time can have a copy of that book on their reader.

This would allow for copyright control, same as for existing hardcopy books and magazines.

And, yes, I know you can have up to five people access the same book on Kindle at a time providing a business library workaround.  I want to make it the real deal with administration, etc.

Get to know your neighbours

In this day and age we have all become a bit weird about getting to know our neighbours. We all seem to be focussed on our work colleagues and old friends.

So how about having a circle like on Google+ that allows people on your street to tweet an invitation.

• “Lord of the Rings marathon on Thursday night at number 22. Bring some beer”
• “Barbecue on Tuesday”
• “Watch the finals, come on over”
• “Help wanted to move some logs in the backyard – sausage sizzle and beer afterwards”
• “Charity bakesale for Spina Bifida on Tuesday. Come over on Monday night and have some fun making cupcakes.”
• “Does anyone have a drill I can borrow?”

Legal/accounting advice by subscription               

There are heaps of day to day issues that we sometimes want to know what our rights are for. So think of this as group buying for essential advice – a kind of insurance.

Pay $250 a year and receive basic advice for up to 2 hours. Also, there would be a FAQ section to a website that could give examples and references for similar questions.

You could do the same for basic medical questions, dental issues, plumbing, house maintenance, etc. In all this, you are not obliged to buy services from the person who provides the advice.

I know that the professionals need to say that each situation is different, but most of the time we are just trying to get our heads around the fundamental issues – i.e. what is it, what does it mean and what are our basic options or next steps.

Fixed price rehab/physiotherapy

 I get injured a lot – so I have been in and out of physiotherapists regularly since I was a teenager.

What really gets under my skin is that you receive close attention the first day, then for the next 6-10 weeks you are going back a couple of times a week just to use their equipment, but you still pay full consultation rates for at most 30 seconds of personal time with the physio on these visits.

So, how about packaging it up so that there is a one off fixed price for complicated injuries. You get attention when you need it, and you can come back as often as you like to use equipment till you are fixed. If you really need ultrasound or something, why can’t there be on call technically trained staff who don’t need a full college education.

Turn the whole experience into a production line and pump them through. It might feel impersonal but in many ways it is more honest, cheaper for the patient, and most of all it is more likely to get the person to stay all the way through.

A single contacts list

In the smart phone and tablet world I would love to have a contacts list that I can use on any device anywhere. Even on any software anytime.

Ideally it would be totally independent of technology or operating system. It would be something I could have for life, and it would be regularly backed up.

I was recently traumatised by having my first mobile phone virus (via MMX) and this has really got me thinking about this.

In case you are thinking that this already happens – well, yes, but not really. Google and Apple don’t talk that well. Business card contact software is great, but everything has to translate to everything else, but database fields don’t always correlate. I have been juggling things between Blackberry, Google Apps, Android, iPhone, iPad, Outlook and Cardscan. It is making me go nuts.

Can we have a universal standard out there please.

Imagine receiving a call from someone and it would automatically upload their name, job title, contact email and address into your phone. It could be placed in a temporary group (circle) and you can then decide to keep it or not later.  As the person making the call you can provide a certain amount of information based on whether this is a query about a product, a call to a friend or a business call.

Let’s automate contacts folks.

Hot Desking Brokerage

 Just like AirBnB is helping people offer up their spare bedroom as accommodation to strangers, how about we advertise and offer up spare desks around town for people.

I can tell you who I want in my office – graphic designers, web designers, IT types, and all sorts of other freelancers. I’d be willing to rent them a desk for a month or year at a time at a much cheaper rate than they could otherwise get.

Yes, there are legal issues, but these are solvable.

Get rid of universities – give us proper online training

 I don’t mean this literally. What I mean is that there is a tendency to overcomplicate everything now.

A while ago I helped clean up my grandfather’s library after he passed away. I couldn’t believe how thin textbooks were. He studied business and accounting at night school in the 1930’s and when I read the content not much had changed except for the thickness of the texts.

Nowadays it almost seems like if you want to learn how to change a band aid you need to train how to be a qualified surgeon.

Let’s get back to the fundamentals, remember that until the 90’s university was the exception and not the rule, and recognise that vocational training is all most people need most of the time.

A lot of the online content nowadays is video clips. Sorry, but this isn’t really training. Mix it up a bit with readings, problem solving, essay writing, submit your own videos, etc.  Make learning and training part of our everyday lives and make it easy.