Tuesday, 5 January 2016

HSBC Online Banking Down

HSBC Online Banking is Down

As of 9:47 on 5th January, 2016, HSBC UK's Online Banking service has been down for AT LEAST 18 hours.  Last night, there was NO service (I was getting 500 errors).  This morning, still no service, but this is due to "a higher volume of traffic than usual".


Wow, HSBC, wow.



Have you heard of "Load Testing" or "Quality Assurance"?  How about "Performance Monitoring"?  I look forward to you VERY PUBLICLY publishing your Root Cause Analysis and Post Mortem, HSBC UK.

Update at 10:00

It looks from Paul85D's screencap from yesterday that HSBC UK are using an Oracle Application server (note the un-configured "you@your.address").  I assume that you are lending them a hand right now, Oracle?

Update at 10:05

DownDetector provide an excellent view of the situation from the user's perspective.  A great demonstration of crowd-sourcing data.  I hope that HSBC UK are a little more proactive than this for fault monitoring, though.

Update at 10:22

The BBC News guys are on to it.  This will, of course, just result in anyone and everon who has an HSBC account trying to log in to see if their service is OK.

Short term: "Damn.  Now no-one will be able to connect for days."
Long term: "Good.  This will kick HSBC into action to a) sack whoever caused this nightmare and b) treat their Internet Banking service team a lot better."

Update at 10:30

Twitterfall is a great way to watch HSBC's nightmare unfold.

Update at 10:34

Unrelated, but this video give a glimpse of how HSBC UK IT might look right now (from "All of our data is GONE!"):

Update at 11:11

You can track the HSBC Share Price here

Update at 11:13

You can complain online here: https://www.hsbc.co.uk/1/2/contact-and-support/feedback-and-complaints .  At least that part of HSBC's website is working right now.

Update at 11:23

Xhalr is something that will help all HSBC customers trying to use online banking right now:

Update at 11:25


Update at 11:30

...and it's back! (applies to Business banking, though running a little slowly)

Update at 11:49

I spoke too soon.  This is what I get when I log in:

Update at 11:52

OK, so THIS is new.  DNS failures now...  Perhaps they're reverting back to the old system?


Update at 12:00

Yes, Business Banking is back, and looking very much like the old system to me.  I'll sign off now, but will update further if appropriate.

Update at 12:12

ARGH!  Started making payments and it's back down again.

Update at 13:12

Still down.  Apologies have been sent to all those my company owes money to.

Update at 13:25

Back up for now.  I expect HSBC IT people are now staring intently at their screens with fingers crossed.

...and tomorrow... the finger pointing starts...

Update at 16:49

One thing worth pointing out.  The relevant A record has a TTL of (randomly) between 0 and 20 seconds.  OUCH.  This is REALLY not good.  HSBC, are you SURE people need to re-fetch an authoritative answer with each query?  Also, are you SURE that this is not an attack?  I sincerely hope that you are in control of that DNS record, because if you are not, customer traffic could potentially be re-routed elsewhere on the Internet.


Update at 17:11

There's an interesting discussion about DNS going on at The Register.  Also, a lot of HSBC abuse from unhappy customers.

25 comments:

  1. Great and useful article. Creating content regularly is very tough. Your points are motivated me to move on.


    SEO Company in Chennai

    ReplyDelete
  2. Super post! Really nice info, thank you!
    --- Stick war ---

    ReplyDelete
  3. Well Said, you have furnished the right information that will be useful to anyone at all time. Thanks for sharing your Ideas. Get More Info

    ReplyDelete
  4. Those with a supposedly better credit score are going to have a lower interest, and investors are going to earn less money off of the interest when this particular borrower pays off their loan.
    guarantor loans

    ReplyDelete
  5. The essential theory given is respectable. Can't be any better.
    Changelly

    ReplyDelete
  6. There is noticeably a bundle to learn about this. I assume you made sure good points in features also. second chance bank accounts colorado

    ReplyDelete
  7. Fantastic guide! Chatting about how treasured the entire digesting. I hope to learn from way more away from you. I know you’ve awesome look and additionally view. I happen to be very much contented using data. https://www.set-up-company.com/available-services/china-incorporation-bank-account-opening/

    ReplyDelete
  8. Amazing article. Your blog helped me to improve myself in many ways thanks for sharing this kind of wonderful informative blogs in live. I have bookmarked more article from this website. Such a nice blog you are providing ! Kindly Visit Us @ Best Travels in Madurai | Tours and Travels in Madurai

    ReplyDelete
  9. This comment has been removed by the author.

    ReplyDelete
  10. It can be difficult to write about this subject. I think you did an excellent job though! Thanks for this! https://www.bwgfinance.co.uk/

    ReplyDelete
  11. Hi just wanted to give you a quick heads up and let you know a few of the pictures aren’t loading properly. I’m not sure why but I think its a linking issue. I’ve tried it in two different internet browsers and both show the same results. California mortgage rates

    ReplyDelete
  12. It is a good site post without fail. Not too many people would actually, the way you just did. I am impressed that there is so much information about this subject that has been uncovered and you’ve defeated yourself this time, with so much quality. Good Works! Sistema de Conciliacion Bancaria

    ReplyDelete
  13. In the 1980s, the Reserve Bank focused on "improvements in the productivity" of the banking sector. Being convinced that technology is the key for improving in productivity, the Reserve Bank took several initiatives to popularize usage of technology by banks in India.Bank in India

    ReplyDelete
  14. It’s in point of fact a great and helpful piece of info. I am happy that you shared this useful information with us. Please stay us up to date like this. Thanks for sharing. checkbook

    ReplyDelete
  15. When you are travelling to a new city for the first time, the first thing you need to assure in advance is that you have reliable airport taxi service that will assist you to reach your desired location on time and without any hassle. Heathrow taxi seek to serve all travellers in the best possible way, guaranteeing a comfortable journey. But for this, it is very essential to choose a taxi rental company that is not only reliable but affordable as well. Here at Ride Executive, we have been providing professional and prompt taxi facilities to all the travellers at the nominal charges for many years. No matter whether you are travelling to London and Buckinghamshire for business conference or family trip, you can book our car to make your trip fully comfortable and secure. We have a great variety of vehicles ranging from executive cars and MPVs, and hence we can cater almost all types of travelling needs. Our drivers are well-trained, experienced, honest and friendly people. They are always on time and treat their customers in the best possible manner. Unlike many other car rental companies, we do not charge high for our services, thus people with low budget can also avail our services without concerning about their budget. The best thing is you can pay your bills through major credit cards like American Express cards, PayPal card and so on. To get more information about our executive cars transport Services and get taxi quote online visit our website or feel free to contact us. We are always here and ready to assist our valued customers. You can also explore our official site to get more details about our services. Airport transfers

    ReplyDelete
  16. Hello I am so delighted I located your blog, I really located you by mistake, while I was watching on google for something else, Anyways I am here now and could just like to say thank for a tremendous post and a all round entertaining website. Please do keep up the great work. personal loan | standard chartered bank credit card

    ReplyDelete
  17. This comment has been removed by the author.

    ReplyDelete
  18. The countries have given the right of issuing money to private bankers. In turn, these private central banks lend the states with interest and therefore, have economic and of course, political power. https://myifsccode.com/canara-bank-ifsc

    ReplyDelete
  19. When you are travelling to a new city for the first time, the first thing you need to assure in advance is that you have reliable airport taxi service that will assist you to reach your desired location on time and without any hassle. Heathrow taxi seek to serve all travellers in the best possible way, guaranteeing a comfortable journey. But for this, it is very essential to choose a taxi rental company that is not only reliable but affordable as well. Here at Ride Executive, we have been providing professional and prompt taxi facilities to all the travellers at the nominal charges for many years. No matter whether you are travelling to London and Buckinghamshire for business conference or family trip, you can book our car to make your trip fully comfortable and secure. We have a great variety of vehicles ranging from executive cars and MPVs, and hence we can cater almost all types of travelling needs. Our drivers are well-trained, experienced, honest and friendly people. They are always on time and treat their customers in the best possible manner. Unlike many other car rental companies, we do not charge high for our services, thus people with low budget can also avail our services without concerning about their budget. The best thing is you can pay your bills through major credit cards like American Express cards, PayPal card and so on. To get more information about our executive cars transport Services and get taxi quote online visit our website or feel free to contact us. We are always here and ready to assist our valued customers. You can also explore our official site to get more details about our services. Airport transfers

    ReplyDelete
  20. Thanks for the blog filled with so many information. Stopping by your blog helped me to get what I was looking for. Now my task has become as easy as ABC. 상품권 현금화

    ReplyDelete
  21. I think that thanks for the valuabe information and insights you have so provided here. One Pearl Bank

    ReplyDelete
  22. The quicker you can move the cash the better. So you need to put that $100 into a venture and remove $100 from it at the earliest opportunity while leaving the underlying $100 in the underlying speculation. Money Geek

    ReplyDelete
  23. "Finances represent the formation of centralized ad decentralized money sources, economical relations relatively with the distribution and usage, which serve for fulfillment of the state functions and obligations and also provision of the conditions of the widened further production". free money

    ReplyDelete