Qlik here for the answers.

I had a meeting at Viridian yesterday regarding Stock Rationalisation and they were showing me a very impressive in-house developed ‘GisWeb’ tool they’re using to manage their stock. The meeting got me thinking and so now we’re writing an algorithm that will tell us which of your homes are outliers and then show you best stock swap matches – the rest is then up to you.

They also told me they use Qlikview which was of interest as my next meeting of the day was to meet up with friends who are Qlik Partners. We’ve been working with Qlik for a few years and my friends actually came out to our office in Nagpur to give Qlik Training when we first started with them. If you haven’t used Qlik then you can download a free ‘personal’ version called QlikSense but between my two meetings the penny finally dropped. It doesn’t matter how much data I supply, if you can’t do anything with it then it just sits in the ‘umm_quite_interesting’ folder on your PC along with all the other data you’re not too sure what to do with.

This is Expenditure on Repairs against Group Stock.

Housing Associations Contracts with Local Authorities (click to enlarge)
Housing Association Contracts with London Local Authorities

Land Banking – excluding Government who owns the most Land Registry ‘Land’ titles? (click to enlarge)

Land Banking – HAs and LAs who own the most Land Registry ‘Land’ titles? (click to enlarge)

So here’s what we’ll do, if you have a licence to our data and need a graph, chart, pivot table, KPI report etc. built using any of our data sets, put on a PDF, and sent to you we will do this free of charge as many times as you like. If you have other data you would like us to look at, try us, and we’ll see what we can do.

We’re adding new data sets to our Data Bank every day so you can pick and choose what data you need for your reports.

I hope you have a great day.

Phew, what a scorcher!

Happy Monday everyone, I’m back in Nagpur and it looks to be another scorcher today although not the 46.6 degrees we had at the weekend. If I was an ice-cream this is what I would look like ….


… but thankfully I’m not an ice-cream so I just look like a sweaty Brit.
If you don’t know the story I set up HousingNet in 1997 after doctors told me I could no longer be a scuba diving beach bum having had a second back operation. I’ve since had another but we’re all good now that I have a metal rod holding everything together.

I had come back to the UK from the States where a Texan friend Jim, who I met in Australia, and I were knocking out a fake Caesars Palace perfume set outside Caesars Palace in Vegas. We were living in the Treasure Island Casino at the time having left LA because of the earthquake. I’d been away since 1986 (joined the REME then went on a world tour when I realised I was a rubbish mechanic) and arrived back in London where my brother was the Manager of Calthorpe who had the repairs contract for Notting Hill Housing. They gave me a job and let me drive the van to pick up rubbish – it was brilliant as I was re-selling 25% of everything that was being thrown out – take note – recycling pays. So I had the Laminectomy in 95, took ages to recover and moved to New Zealand as I had picked up a residence permit along my travels, came back, moved to Liverpool and bought a computer to turn on, make a square in ‘Paint’, erase it and then turn it off again. I had an idea to start an after-hours call handling service as we, my brother and I, knew we could fix a lot of the jobs over the phone and started ‘Com1 UK’. Hang on, Chris and I, doing back to back 12 hour shifts on the phone, time for a re-think. Remember the Housing Association Year Book? I do. I learnt how to build a web page and put all Housing Association addresses online and called it HousingNet.

I came out to India in 1999 as I had ideas and no money. I ended up in Nagpur, met Ajay, rented a flat, started an office and here we all are today harvesting Housing data. I’m based in London but my office and nearly all my staff are here in Nagpur.

So what do we do? We have built our reputation on having very up-to-date data on Senior Managers working in Housing Associations and Local Authorities. We collect just about everything you put on the web that would be of interest to the wider community and the business users who are our main clients. At the beginning of this year we started collecting planning applications with a particular interest in ‘housing’ oddly enough. By the middle of June we will be more up to date than any other planning application resource in the UK – eat my shorts Barbour!

In January a client of ours told us that if we could supply tender data they would pay us lots more money for the licence so we started looking in to it. Initially I thought it was simple. OJEU supply the tenders, we take all the historical data for free and pick up the daily ones as they come out – easy money! How wrong I was. People, if you’re involved in tenders you have my admiration / sympathy. What a nightmare! We’re programmers here so we work with very large data sets and OJEU is like a story where the beginning sounds really exciting, the middle is always missing and the end is like watching the last episode of Undercover on repeat – it makes no sense.

So I’m here, in the Orange City, in the height of Summer, sorting it out. We’re collecting all Contract Registers, harvesting every tender from all Procurement Portals, all Local Authority and, where available, HA expenditure sheets, LA cabinet minutes, news stories relating to contracts and anything else we can get our hands on. So the question is, what do you want to know and most importantly would you pay for it?

Our clients include: Sovereign, ForVivia, Orchard, Capita, United Utilities, The Bank of Tokyo, Paragon, Viridian, Bracknell Forest, BDO, Mobysoft, Willmott Dixon, Aldwyck, CHIC, Kier, SSE, BSkyB, Argos, BMG, British Gas, Lovell, Northgate, Mountjoy, Campbell Tickell, Civica, Orbit, THFC, VINCI … I could go on but you get the idea.

I’m back in the UK on Saturday and after 19 years of hiding behind email I now actually go and visit people if they invite me. So drop me a line if you would like to know more about what we do and how we can help you.

Hope you all have a great week.


Why we built our own UK Planning Application Database

I had a meeting with a client at the beginning of the year to talk about obtaining OJEU tenders. I’ve stayed away from them in the past until I realised what other companies are charging for the service and that you can obtain the raw data for free. We’ve since worked our way through the TED database and collected 91,000 tenders of interest which we’re now organising and live tenders will be on HousingNet by the end of this week.

But I digress …. at the meeting we also discussed obtaining data on new build homes. There is data out there, but again, very expensive so we decided the only way to do it properly was to collect and analyse every UK planning application. We built software to manage the processes and have a team of six who enter the information on our system, isolate records based on key words and update every application of interest.

We are particularly interested in New Build, Conversions, Basements, Lofts and Extensions. We collect data on Office 2 Home, HMO’s, Student Housing, S106, Applicants, Agents and Decisions and so far we’ve collected 94,796 applications, added additional data to 8,406 of them and recorded 49,755 new homes.

The data in itself is fascinating: from seeing which Housing Association has the most applications refused to the number of applications relating to different tree types – Oak 54, Beech 24, Conifer 16 and the ever popular Thuja 4.
The stats for February show Broadland District Council accepting applications for 3535 new homes and Cornwall County Council being the busiest with the highest number of accepted (88) and refused (37) applications.

Our priority was new build but the more we looked at the data the more possibilities we found. Living in South London as I do we are surrounded by basement and loft conversions so we started pulling out this data along with ‘extensions’. Companies are currently using this data to look for marketing opportunities in areas where planning applications are more likely to be accepted, agent league tables and reasons for refusal.

We can compare this data to the 13.2M corporate property titles we have and the 4.2M Social Homes we’ve isolated from that data set. We also use House Price/Sale data and Companies House data to help clients understand where opportunities lie.

Tomorrow we’ll be putting the planning search online and making it free to use for a while as we would welcome your feedback.

If you haven’t used HousingNet before it works like this – we collect data faster than we can get it online so clients purchase a licence to access our online data, maps, tables and exports and then if there is any combination of data or analysis they can’t export from the site we offer a 24 hour turnaround to deliver any bespoke data set free of charge.

HousingNet development is driven by our client’s requests, if you want it we’ll do our best to get it and deliver it in the format you need. View our data packages here.

Social Homes valued at over £500,000

Happy New Year all.

Now that we’ve valued nearly all social homes in England we can start interrogating the data and we’re open to suggestions on this. We welcome your input so please let us know your ideas and we’ll trawl the data to find the answers.

We have a breakdown of properties valued at £300K – £499K and over £500K that we’ve added to the site for our registered users. The data is broken down between Housing Associations and Local Authority stock and then Freehold / Leasehold and Houses / Flats. This snapshot of the data is showing the total homes from HAs and LAs valued at >£500,000 and their total value.

District Total Homes Total Value
CITY OF WESTMINSTER 10507 10,631,697,106
KENSINGTON AND CHELSEA 9814 9,973,303,639
LAMBETH 12284 9,260,658,437
CAMDEN 7266 6,399,516,467
ISLINGTON 8451 6,381,363,664
HAMMERSMITH AND FULHAM 6651 5,986,773,492
HACKNEY 6269 4,447,997,781
WANDSWORTH 5597 4,137,797,080
TOWER HAMLETS 5227 3,422,338,031
SOUTHWARK 4344 2,857,326,031
RICHMOND UPON THAMES 2906 2,538,986,321
BRENT 1994 1,335,670,245
BARNET 1286 1,113,373,236
LEWISHAM 1719 1,075,060,804
EALING 1352 1,065,367,650
HARINGEY 1510 1,000,692,089
MERTON 1023 689,202,489
ELMBRIDGE 594 441,597,491
GREENWICH 649 433,177,005
BROMLEY 559 388,945,931
SEVENOAKS 564 361,506,474
HILLINGDON 349 353,061,921
HOUNSLOW 437 314,167,078
CAMBRIDGE 408 283,629,516
WAVERLEY 414 258,471,262
CROYDON 323 202,483,812
SOUTH BUCKS 242 197,012,613
HARROW 297 196,595,687
TUNBRIDGE WELLS 307 187,973,986
WOKING 276 183,555,334
EAST HERTFORDSHIRE 284 178,851,312
CORNWALL 306 178,224,981

To calculate stock and value we use a combination of the Land Registry Corporate database, NROSH, House Price data and an awful lot of data scraping and cleansing.

If you have an idea for data analysis please send it over. We will share all the output with you whether or not you are registered on Housingnet.

Every Housing Association, on purchasing an unlimited licence from us, can access all their own Land Registry titles including geo-co-ordinates along side our normal data service.

House Price Valuation

We’ve been working with the Land Registry Corporate data set for a couple of years and the bulk of that time has been spent cleansing the data and adding geo-positioning co-ordinates for each social home. In May this year we started to look at portfolio valuation but the lack of bedroom data meant that there was a lot of guess work involved and the results didn’t stand up to scrutiny.

We set about collecting bedroom data and along the way we amassed a large amount of information on individual properties which 6 members of the team add to every day. Through this ‘big data’ exercise we started to get interest from Estate Agents and property websites resulting in an exclusivity licence with www.loyaltystreet.com and other opportunities for data cleansing and analysis.

Property Ownership Midmoor Road SW12
Property Ownership Midmoor Road SW12

Housing Associations own a vast amount of property and not all of it is social housing so we removed everything that wasn’t a flat or a house. We spent months cleansing incorrect road names, Postcodes, door numbers and house/flat designations and wrote countless algorithms to tweak the process each time.

1. Firstly we removed Land Registry Titles that was not a home – examples of this are – student/nursing accommodation, electricity sub-stations, shops, business units and even a bowling alley.
2. The initial property valuation exercise looked at any properties sold since 1995 ie. exact matches from ‘House Price Data’.
3. We then used various criteria to segregate the homes as either Flats or Houses and attached bedroom data to each record.
4. CAGR (Compound Annual Growth Rate) was used to calculate the current price of the property.
5. Based on the initial match with Price Paid data and the number of bedrooms we took out the average price for flat and house based on the Lower Layer Super Output Area or Postcode level, whichever was more accurate.
6. Using the average price we attached a value to the remaining properties and then used the House Price Index to cross check our results.

Finally, last Wednesday, we got something we were happy with for L&Q

Total Social Housing homes: £18.12 billion
Freehold : £15.67 billion
Leasehold : £2.45 billion

It’s not exact but it is, we believe, a very good indication of the current market value.

Currently we’re collecting every Social Housing new build including room sizes, client, contractor, construction type etc. Calculating valuation on 4.6 million social homes and enhancing the data of 3000 homes a week.

Every Housing Association, on purchasing a multi-user licence from us, can access all their own Land Registry titles including geo-co-ordinates along side our normal data service.

What’s new on HousingNet?

We have just recently launched a searchable database of the Land Registry Commercial & Corporate Ownership data set which we’ve set up on https://cloud.google.com/. Amazingly Google Cloud is free for the first year but with no support the set up did have us scratching our heads for awhile. But now we have 13.2M records to search through and the results are very interesting and it’s not all about Social Housing. Did you know that Oxfam own 476 shops, Cancer Research 473, British Heart Foundation 662 and WH Smith only 273 which includes their hospital shops.

Using the Land Registry Data from 2014 and 2015 we’ve been looking at the movement of property titles with regard to how many new titles were added and how many were lost – this will include Right to Own, Shared Ownership and any other private sale. How many titles were transferred to another corporate organisation and how many new titles were transferred from another corporate. Over the next couple of weeks we will be looking at how these figures compare with the newly released RSR for 2015.
View the stock table here.

Housing stock v Land Registry Data comparison

Land Registry are relaxing the rules on the National Dataset licence usage which now means we can share title data with our corporate clients – as long as we pay them more obviously. What this essentially means to you/us is we can now cross reference your data with the National Dataset.

Having been cleaning the Social Housing titles in the Dataset for the past year we now have access to better quality data than Land Registry themselves and here are a few examples of work we can now carry out:

1. We can now compare Housing Association stock data with the National Dataset to make sure your titles are in order. We are offering this as a free service to Housing Organisations who have an active multiple users HousingNet licence.

2. If the product or service you are offering is for private home owners only you can cut down on your direct marketing costs by having us remove all Social Housing and Commercial titles from your mailing lists making you huge savings on direct mail.

3. You might have a database of 100,000 addresses you need us to check or require a monthly update. Whatever your requirements please write or call 07779 715 605 for a chat.

Social Housing in Manchester mapped

Click for full size.

We have finished adding all the Land Registry title data for Social Housing stock in Manchester taking our ‘HousingNet Homes Totaliser’ to 1,536,023 homes mapped. Of course we’ll have to start all over again as these will probably all be sold in the latest Right-to-Buy shenanigans.

Hopefully we will have our league table of Housing Association portfolio values completed by the end of next week which uses Land Registry titles and House Price data + regional annual increase. If any Housing Association would like us to upload their actual stock data so they can compare it with Land Registry data, get a more accurate portfolio value and access all the tools and data on HousingNet please get in touch – we are happy to do this free of charge to any of our registered clients.

New HousingNet Map is open for business

The new map work has gone live and we’ve opened up some access for non-registered users such as crime, brownfield and flood plain data but the housing stock and Land Registry data is locked unless you’re a client of ours.
The next stage is looking at stock valuation based on house price sale data so we can look at portfolio values and we’re also working on an algorithm which will highlight high instances of crime / high density Social Housing for each Housing Association.

Visit the map

As always we are very open to ideas.

The new map is coming ….. honestly.

HN_Map_Area HN_Map_Homes
So the first map was a trial run, we didn’t really know how things were going to turn out but once we saw it live and listened to feedback we set about building a new one. Apologies to the people I told it would be going live at the beginning of April, it isn’t, Samir tells me it’s two weeks away.
We’ve added Crime, Land Registry Land Titles, Brownfield sites, Flood areas, Broadband, Green Belt, Social and Private Rent by Ward and House Price data.
After our rather scatter gun approach to adding Land Registry title data we’re now concentrating on one city at a time. Liverpool is complete, Manchester is nearly done then we’re moving down to Birmingham and on to London.

Our next project for Registered Users is to rebuild the Export Panel where we’ll enable every field of data we collect available for export. When we do this we’ll remove the Housing Stock and Financial Data from the Housing Association pages which will speed up the load times as we know this can be frustrating.

We welcome any suggestions and do let us know if you have any requests as we have really enjoyed working with a Housing Association recently which resulted in new fields added to the export.