Category Archives: HousingNet Blog

Why we built our own UK Planning Application Database

I had a meeting with a client at the beginning of the year to talk about obtaining OJEU tenders. I’ve stayed away from them in the past until I realised what other companies are charging for the service and that you can obtain the raw data for free. We’ve since worked our way through the TED database and collected 91,000 tenders of interest which we’re now organising and live tenders will be on HousingNet by the end of this week.

But I digress …. at the meeting we also discussed obtaining data on new build homes. There is data out there, but again, very expensive so we decided the only way to do it properly was to collect and analyse every UK planning application. We built software to manage the processes and have a team of six who enter the information on our system, isolate records based on key words and update every application of interest.

We are particularly interested in New Build, Conversions, Basements, Lofts and Extensions. We collect data on Office 2 Home, HMO’s, Student Housing, S106, Applicants, Agents and Decisions and so far we’ve collected 94,796 applications, added additional data to 8,406 of them and recorded 49,755 new homes.

The data in itself is fascinating: from seeing which Housing Association has the most applications refused to the number of applications relating to different tree types – Oak 54, Beech 24, Conifer 16 and the ever popular Thuja 4.
The stats for February show Broadland District Council accepting applications for 3535 new homes and Cornwall County Council being the busiest with the highest number of accepted (88) and refused (37) applications.

Our priority was new build but the more we looked at the data the more possibilities we found. Living in South London as I do we are surrounded by basement and loft conversions so we started pulling out this data along with ‘extensions’. Companies are currently using this data to look for marketing opportunities in areas where planning applications are more likely to be accepted, agent league tables and reasons for refusal.

We can compare this data to the 13.2M corporate property titles we have and the 4.2M Social Homes we’ve isolated from that data set. We also use House Price/Sale data and Companies House data to help clients understand where opportunities lie.

Tomorrow we’ll be putting the planning search online and making it free to use for a while as we would welcome your feedback.

If you haven’t used HousingNet before it works like this – we collect data faster than we can get it online so clients purchase a licence to access our online data, maps, tables and exports and then if there is any combination of data or analysis they can’t export from the site we offer a 24 hour turnaround to deliver any bespoke data set free of charge.

HousingNet development is driven by our client’s requests, if you want it we’ll do our best to get it and deliver it in the format you need. View our data packages here.

Social Homes valued at over £500,000

Happy New Year all.

Now that we’ve valued nearly all social homes in England we can start interrogating the data and we’re open to suggestions on this. We welcome your input so please let us know your ideas and we’ll trawl the data to find the answers.

We have a breakdown of properties valued at £300K – £499K and over £500K that we’ve added to the site for our registered users. The data is broken down between Housing Associations and Local Authority stock and then Freehold / Leasehold and Houses / Flats. This snapshot of the data is showing the total homes from HAs and LAs valued at >£500,000 and their total value.

District Total Homes Total Value
CITY OF WESTMINSTER 10507 10,631,697,106
KENSINGTON AND CHELSEA 9814 9,973,303,639
LAMBETH 12284 9,260,658,437
CAMDEN 7266 6,399,516,467
ISLINGTON 8451 6,381,363,664
HAMMERSMITH AND FULHAM 6651 5,986,773,492
HACKNEY 6269 4,447,997,781
WANDSWORTH 5597 4,137,797,080
TOWER HAMLETS 5227 3,422,338,031
SOUTHWARK 4344 2,857,326,031
RICHMOND UPON THAMES 2906 2,538,986,321
BRENT 1994 1,335,670,245
BARNET 1286 1,113,373,236
LEWISHAM 1719 1,075,060,804
EALING 1352 1,065,367,650
HARINGEY 1510 1,000,692,089
MERTON 1023 689,202,489
ELMBRIDGE 594 441,597,491
GREENWICH 649 433,177,005
BROMLEY 559 388,945,931
SEVENOAKS 564 361,506,474
HILLINGDON 349 353,061,921
HOUNSLOW 437 314,167,078
CAMBRIDGE 408 283,629,516
WAVERLEY 414 258,471,262
CROYDON 323 202,483,812
SOUTH BUCKS 242 197,012,613
HARROW 297 196,595,687
TUNBRIDGE WELLS 307 187,973,986
WOKING 276 183,555,334
EAST HERTFORDSHIRE 284 178,851,312
CORNWALL 306 178,224,981

To calculate stock and value we use a combination of the Land Registry Corporate database, NROSH, House Price data and an awful lot of data scraping and cleansing.

If you have an idea for data analysis please send it over. We will share all the output with you whether or not you are registered on Housingnet.

Every Housing Association, on purchasing an unlimited licence from us, can access all their own Land Registry titles including geo-co-ordinates along side our normal data service.

House Price Valuation

We’ve been working with the Land Registry Corporate data set for a couple of years and the bulk of that time has been spent cleansing the data and adding geo-positioning co-ordinates for each social home. In May this year we started to look at portfolio valuation but the lack of bedroom data meant that there was a lot of guess work involved and the results didn’t stand up to scrutiny.

We set about collecting bedroom data and along the way we amassed a large amount of information on individual properties which 6 members of the team add to every day. Through this ‘big data’ exercise we started to get interest from Estate Agents and property websites resulting in an exclusivity licence with and other opportunities for data cleansing and analysis.

Property Ownership Midmoor Road SW12

Property Ownership Midmoor Road SW12

Housing Associations own a vast amount of property and not all of it is social housing so we removed everything that wasn’t a flat or a house. We spent months cleansing incorrect road names, Postcodes, door numbers and house/flat designations and wrote countless algorithms to tweak the process each time.

1. Firstly we removed Land Registry Titles that was not a home – examples of this are – student/nursing accommodation, electricity sub-stations, shops, business units and even a bowling alley.
2. The initial property valuation exercise looked at any properties sold since 1995 ie. exact matches from ‘House Price Data’.
3. We then used various criteria to segregate the homes as either Flats or Houses and attached bedroom data to each record.
4. CAGR (Compound Annual Growth Rate) was used to calculate the current price of the property.
5. Based on the initial match with Price Paid data and the number of bedrooms we took out the average price for flat and house based on the Lower Layer Super Output Area or Postcode level, whichever was more accurate.
6. Using the average price we attached a value to the remaining properties and then used the House Price Index to cross check our results.

Finally, last Wednesday, we got something we were happy with for L&Q

Total Social Housing homes: £18.12 billion
Freehold : £15.67 billion
Leasehold : £2.45 billion

It’s not exact but it is, we believe, a very good indication of the current market value.

Currently we’re collecting every Social Housing new build including room sizes, client, contractor, construction type etc. Calculating valuation on 4.6 million social homes and enhancing the data of 3000 homes a week.

Every Housing Association, on purchasing a multi-user licence from us, can access all their own Land Registry titles including geo-co-ordinates along side our normal data service.

What’s new on HousingNet?

We have just recently launched a searchable database of the Land Registry Commercial & Corporate Ownership data set which we’ve set up on Amazingly Google Cloud is free for the first year but with no support the set up did have us scratching our heads for awhile. But now we have 13.2M records to search through and the results are very interesting and it’s not all about Social Housing. Did you know that Oxfam own 476 shops, Cancer Research 473, British Heart Foundation 662 and WH Smith only 273 which includes their hospital shops.

Using the Land Registry Data from 2014 and 2015 we’ve been looking at the movement of property titles with regard to how many new titles were added and how many were lost – this will include Right to Own, Shared Ownership and any other private sale. How many titles were transferred to another corporate organisation and how many new titles were transferred from another corporate. Over the next couple of weeks we will be looking at how these figures compare with the newly released RSR for 2015.
View the stock table here.

Housing stock v Land Registry Data comparison

Land Registry are relaxing the rules on the National Dataset licence usage which now means we can share title data with our corporate clients – as long as we pay them more obviously. What this essentially means to you/us is we can now cross reference your data with the National Dataset.

Having been cleaning the Social Housing titles in the Dataset for the past year we now have access to better quality data than Land Registry themselves and here are a few examples of work we can now carry out:

1. We can now compare Housing Association stock data with the National Dataset to make sure your titles are in order. We are offering this as a free service to Housing Organisations who have an active multiple users HousingNet licence.

2. If the product or service you are offering is for private home owners only you can cut down on your direct marketing costs by having us remove all Social Housing and Commercial titles from your mailing lists making you huge savings on direct mail.

3. You might have a database of 100,000 addresses you need us to check or require a monthly update. Whatever your requirements please write or call 07779 715 605 for a chat.

Social Housing in Manchester mapped

Click for full size.

We have finished adding all the Land Registry title data for Social Housing stock in Manchester taking our ‘HousingNet Homes Totaliser’ to 1,536,023 homes mapped. Of course we’ll have to start all over again as these will probably all be sold in the latest Right-to-Buy shenanigans.

Hopefully we will have our league table of Housing Association portfolio values completed by the end of next week which uses Land Registry titles and House Price data + regional annual increase. If any Housing Association would like us to upload their actual stock data so they can compare it with Land Registry data, get a more accurate portfolio value and access all the tools and data on HousingNet please get in touch - we are happy to do this free of charge to any of our registered clients.

New HousingNet Map is open for business

The new map work has gone live and we’ve opened up some access for non-registered users such as crime, brownfield and flood plain data but the housing stock and Land Registry data is locked unless you’re a client of ours.
The next stage is looking at stock valuation based on house price sale data so we can look at portfolio values and we’re also working on an algorithm which will highlight high instances of crime / high density Social Housing for each Housing Association.

Visit the map

As always we are very open to ideas.

The new map is coming ….. honestly.

HN_Map_Area HN_Map_Homes
So the first map was a trial run, we didn’t really know how things were going to turn out but once we saw it live and listened to feedback we set about building a new one. Apologies to the people I told it would be going live at the beginning of April, it isn’t, Samir tells me it’s two weeks away.
We’ve added Crime, Land Registry Land Titles, Brownfield sites, Flood areas, Broadband, Green Belt, Social and Private Rent by Ward and House Price data.
After our rather scatter gun approach to adding Land Registry title data we’re now concentrating on one city at a time. Liverpool is complete, Manchester is nearly done then we’re moving down to Birmingham and on to London.

Our next project for Registered Users is to rebuild the Export Panel where we’ll enable every field of data we collect available for export. When we do this we’ll remove the Housing Stock and Financial Data from the Housing Association pages which will speed up the load times as we know this can be frustrating.

We welcome any suggestions and do let us know if you have any requests as we have really enjoyed working with a Housing Association recently which resulted in new fields added to the export.