IoT Podcast Logo

In S2 episode 26, Tim Winters- CTO at QA Cafe tells us more about the testing that goes into an IoT product📲, from the processes of testing, the barriers and the security aspects for different use cases🔒

Sit back, relax, tune in and be the first to discover…

  • Tim’s background 💥
  • What does QA Cafe do? 💥
  • Where is the Smart Home industry at currently and what are the barriers when it comes to testing? 💥
  • How is the QA Cafe Passport Automated IoT helping increase the reliability of smart home devices? 💥
  • What are the processes involved when it comes to testing? 💥
  • How can we improve the interoperability of devices? 💥
  • How is testing helping businesses accelerate IoT deployments? 💥
  • Will there be more innovation when it comes to testing IoT devices in the future? 💥

And much more!

ABOUT THE GUESTS

Tim Winters is the CTO at QA Cafe, a dynamic software company composed of experts in networking, consumer electronics, and security. They develop industry-leading network device test solutions and network analysis tools for business use while providing customers with world-class support.

Follow Tim on LinkedIn Here

Find out more about QA Cafe and Passport Automated IoT Here 

Episode Transcript

Brad King-Taylor
Hello, and welcome to The IoT Podcast. I’m your host and state Brad King-Taylor. And today I’m lucky enough to be joined by Tim winters, who is the CTO of QA cafe. Before we get into it, don’t forget whatever platform you’re listening or watching on, please like and subscribe to the content, it gives you all the insight into the new episodes and when they release. So Tim, absolute pleasure to have you today. And thank you for joining us. I suppose the best way to start is to jump straight into it. Right. So as I mentioned, it’s great to have you with us. So it what probably the best place to go from here is can you explain the background from yourself how you got into the crazy world that we find ourselves in and your position as a CTO of QA cafe?

Tim Winters
Absolutely. I can talk about all of that. So thanks for having us. It’s really great to be here. And my background is a little bit I’ve been in the networking industry or testing networking industry for over 20 years now. I started at the interoperability lab. And I worked there for about 20 years. And there we did all kinds of testing for things interop in particular, you know, at first, obviously, IoT wasn’t a thing. And then over time, it became way more prevalent. And then by the end of my career, there it was, it was all everyone wants to talk about which one to talk about interop because IoT, you need it. There’s lots of different devices, no one is buying one solution from everybody for all of your IoT needs. So that was a big part of it.

Tim Winters
And then I moved over to QA cafe. I’ve been here for over two and a half years now. And what we’re doing here is we created were working similar thing. We built our testing, we have a test tools that were based on testing home gateways. We have an Analyst tool that does that. And then people were coming to us saying, Hey, you guys have built this for gateways, what do you have in the IoT space. And so that kind of moved us in that direction. It’s a very diverse space, which is fun for us. And that’s kind of how we got involved. These customers were looking for ways to verify their products. And by customers here, we mean people building it, but we also mean people using it, and want to make sure that their stuff is working. Yeah.

Brad King-Taylor
Amazing. So you’ve got an extensive background, 20 years. Very impressive. I, if I did 20 years, I probably have less hair than I actually have, which is madness. So hats off to you. It’s a really good overview of your background. So I suppose for the benefit of the listeners, it’d be good to get an insight of what QA Cafe actually do in terms of the technology and that sort of stuff and what you’re trying to achieve.

Tim Winters
Yeah, absolutely. So we are, we’re a small software company that builds test tools, and tests and analysis tools. So the test tool we have is CD router, it’s been focused on home gateways, we did a lot, we do lots of testing, where we do lots of USP tr 69 management protocols. It’s used by both service providers who, you know, want to check things that they’re getting often, you know, even when they’re getting no device from a manufacturer, they want to put their own spin on it, they want to add their own, they have their own requirements, whatever they might be. So they’re verifying those, and then we work with the manufacturers of it. Um, one of the bigger changes to is the open source community and that space.

Tim Winters
So we’ve got a lot more experience with the RDKs, purple OS is of the world open WRT is of the world that’s been one of the massive changes in that space. And I expect we’ll see similar things in the IoT space over time. So that’s one product that we’ve built. And we’ve always had. And then we have another one called Shark, which is more on the test and analysis side, it’s basically Wireshark. But in the cloud, and we have, we’ve added a lot more analysis tools on that side. So those are our two main shifts. And then we have a new thing that we call passport, that is kind of a blend of the two. So it does the testing and it does the testing of IoT devices, plug them in, you connect to them. And then we have it directly connected to cloud shark for allowing people to analyse look deeper into what their IoT devices doing and a lot of cases, you know, it’s not coming from the individual they want they want to know who it’s talking to and why it’s talking to and that’s where cloud shark comes in.

Brad King-Taylor
Yeah, amazing. So you’ve got quite a broad spectrum in terms of what you’re trying to achieve. But I do want to come back to that the passport later on, because I think what what you’re trying to achieve there is amazing it’s gonna help a lot of people. I want to narrow down slightly in terms of one of the sectors so the smart home industry, where that is at currently and and what barriers you’re facing when it comes to the testing side of things.

Tim Winters
Yeah, so what one of the things we’ve been seeing in the smart home, there’s a couple of changes, or, you know, people have come to us with all these questions, you know, smart home testing, what we found is it’s very manual, that we started talking to companies, they were like, Oh, we have a, we set up a router, we set up this scenario. And then we hook our IoT device in it, it’s at the end of their process, which isn’t good doesn’t follow any of their agile practices where they want it, you know, that it’s, it’s happening at the end, it’s very, like ad hoc, not good. Not repeatable has been a big, that was probably the most common thing we heard. It’s not automated. And it’s not repeatable.

Tim Winters
So they’ll see something, customer reports, something they may have seen it in the lab, that was really, one of the problems we wanted to go out and solve is that manual testing seemed to be a big part of the smart home. So one of the things that we thought at the time was like, Hey, let’s build something that people repeatable, run it over again. Also, oddly enough, we get a lot of requests for legacy stuff. And the reason for that is diversity of networks. And I think of my father, and how he never updates anything in his house. Everybody wants to new thing. So like, you know, I don’t even know what his Wi Fi is, it’s probably something awful like B or G. Who knows? Yeah, exactly.

Tim Winters
What we heard from customers was they were like, great, the new stuff we want to test. But we also want to make sure that it’s going to work in someone’s environment, because you don’t let me be honest, you don’t want my father calling you try it as a support ticket. So that’s one of the things that we’ve built is trying to make that work in a way that you can test both the legacy and the newest stuff and make it repeatable, so that, you know, hey, it doesn’t work at this speed. In particular, you know, the tool goes through all the Wi Fi settings. And that’s the one where you don’t know what the encryption is. It’s a very, home networks are very diverse. And so what we’re trying to make it as repeatable as possible so that you can drill down and find your issue and be like, Okay, we know that this, this is a problem area that we’re going to have to address.

Brad King-Taylor
Yeah, amazing. And I feel like the connectivity is a listen to a radio station in the UK called Talksport. And there’s an advert on that all the time about the anger of certain people, when they lose connectivity in the home or whatever it might be. It’s, it’s very relevant to a lot of people going back to your passport then and the smart home. So how were you helping increase the reliability for the smart home devices using the QA Cafe passport?

Tim Winters
Yeah, so passport is a device it, you know, you hook it up, it acts, it basically emulates a gateway, and we plug it in, it’s on all the time. So it’ll act, you can put it in your house as your gateway. But really, what it does is it goes through all these different scenarios. So it tries link flapping, turning things off and on different wireless settings to make sure the device can connect. And it can deal with all of those scenarios, because a lot of IoT devices need conductivity. They’re trying to connect to a cloud service somewhere that’s keeping track of something whether it’s opening your garage door telling you that you need milk for your your out of milk in your fridge, all of those things need to connect to something else. They’re not self sustaining. And so that’s one of the things we do. We have stability testing, that was a really piece of feedback we got from customers was, you know, great. It connected the first time but what happens eight hours later, is it going off and on? Is it? What things is it doing? And that’s where the repeatability came up. So we built this tool, the idea is definitely more on the automation CI CD side that allow people to repeat testing run on a nightly per commit basis to find out immediately when they broken something not weeks later, is sort of the idea that we’re going for.

Brad King-Taylor
Yeah, amazing. I really like the fact that you off and on, because that’s literally all we ever get told right? When you ring it support or whatever it is. Have you tried turning it off again? Well, of course, because it’s the first thing you’re going to tell me to do. So yeah, exactly that so yeah, really clever. Absolutely. What process are involved when it comes to the testing?

Tim Winters
So yeah, we’ve built what we’ve done here is it’s an individual tool, we’ve put a battery, a PDU a power unit into it. So it’ll bring the device up and down to make, again, for the automation, we don’t want users having to do a whole lot. Also, the tool auto discovers things connected to it to test so that you don’t have to be searching for the MAC address or the web UI of your device. We try to make it as little config as possible, mostly because again, this is the diversity conversation with IoT. Is that what you’re when you’re testing a phone or a smart TV, there are lots of buttons to touch when you’re testing a thermostat there or not. Right. And so some of that we had to deal with was trying to figure out what was the sweet spot for what works for you know, as many diverse devices as we can find, right? Whether it’s a home speaker, or whether it’s a you know, a laptop or a smart TV, you’re gonna be very different. And so what we tried to do is make it auto discover, you know, it displays the device chosen what’s going on and can go off and run all those different scenarios inside of the gateway to emulate you know, service providers, very diverse networks. We have a lot of experience As with them, we have 20 years of testing, working with service providers, we tried to emulate a lot of what scenarios they do so that the IoT devices can be prepared for those situations.

Brad King-Taylor
Fantastic. I really like the the idea about the whole device thing, because every every device really is different, different sensors, different different methods of connectivity. And well, they said different a range of buttons. So one of the issues that is highly talked about is the interoperability of devices. So how, how will we how can we improve the inter interoperability of certain things?

Tim Winters
Yeah, I mean, you know, the idea for this came from I, when I worked at the IOL, we had a consortium or testing service that was poor undergraduate students, connecting these devices manually, right. So they would go get a bunch of we had all these gateways, they would hook them up, set them up, and then have to connect the device to each individual one. And it was painful. It was manual, it was painful. I mean, I’m sure those students loved it, they learned some stuff, but it was tough. And so when I moved over to QA, one of the things I wanted to do was build software to do that, because it to create those interops, scenarios. Note, sure, there’s different hardware differences. But there are certifications for the Wi Fi cert. And in doing the hardware interop, we’re really focused on what I would call the network protocol interrupt, making sure that it can connect in particular, this is where the cloud services come up, we do things like DNS Failover, we want to make sure that it doesn’t lock on to one DNS server in today’s world, you know, networks as outages happen, right. And what you don’t want is to be have an IoT device unable to connect, when it should fail over or roll over, because of different things that happen, you know, the some of the tests, we’re making sure that it gets re addresses, we’ve definitely seen devices that gets an address once. And then that’s it, you have to reboot it before it’ll come back, depending on the network settings. And we also do a lot with ipv6. That’s a one, you know, we definitely starting to hear or encounter, ipv6 only networks. IoT works really great for that, because of all the addresses, it can be complicated, because V six is new, and people don’t understand it. And so we have a lot of test content around that as well, for those people who are interested in that area.

Brad King-Taylor
Yeah, I think that relates to a lot of things, though, in our in our world, it’s kind of comes a lot more complicated when you just don’t understand it. And it can be the most simple of things. So what are the challenges that again, across the whole IoT sector is security? Obviously, there’s a lot of connected devices within IoT. So what are the challenges when it comes to security and testing? And and how are you getting over these?

Tim Winters
Yeah, so you know, we have a couple of, we have a couple of things we’re looking at their security is always, this is always going to be something that’s going to be evolving, right? Whatever we’re doing with testing security, it’s not going to be standing solid. So first thing you have to kind of get inside your head is how are we going to change this? What are we going to do? How are we going to make this make sense? For everybody what we’re doing on the security side, you know, there’s different levels, we’re looking into things like bootstrapping, we have other tools or analysis tools, we allow us to the cloudshark side, we have sericata built in. So we can do things like passwords in the clear, you can check those kinds of things, which on an IoT device, you know, might might happen, I think, in the future, you know, talking about the IoT space as a whole. And there’s going to be a lot more about, you know, secure firmware updates, those kinds of things. Because A, I think it needs to be open, you know, the, it’s not great when you buy something, and then it has a security flaw, but that company has gone away, and you have no way to fix it. So do you have to throw it in the trash? What what do you do with this thing that now might be insecure, like? And so I think there’s going to be some really interesting stuff coming out of standards groups or organisations to try to solve some of those problems, which I think will be things that we’re always going to be on top of watching, trying to create test content to make it you know, there’s the classic protocol stuff, is it encrypted? Is it not encrypted? We have a lot of checks for that stuff, hey, this is not, you know, we do Port Scans, those kinds of things to make sure the device is doing what you want.

Tim Winters
Like I said, we have the cloudshark thing, a lot of our customers just run a captcher and see what they’re how much you’d be surprised what these devices are generating and how rapidly they create stuff. And so I think that’s where customers want to understand in particular, you know, we have users who don’t create the IoT device, right? They’re grabbing something off the shelf from somebody else, whether it’s part of their solution, or, you know, service written the service providers case, what’s happening to them is the customers buying something from a retail shop and bring it into the house, very different, but who do they call, they call the service provider. They don’t call the individual company, they’re like, Hey, this isn’t working. It’s obviously your network. They want to know what kind of scenarios they can be like, no, no, no, that thing works, or no, we’ve had problems and here’s, you know, here’s what you have to do to your network. And those are the things on the security side. You know, security has to be good. And it has to be easy because otherwise, people will turn it off. Or be like, Oh, that’s causing me pain and That’s not what we want. So we’re trying to kind of this is where we’re trying to kind of help the industry to find testing that can work, verify these things without too much pain. Because otherwise we’re going to be in trouble as a as an industry, and that people will turn it off. And that’s not what we want.

Brad King-Taylor
Yeah, for sure. And I feel like some of the most successful people that I speak to is all about, especially with security, because like you said, it’s always evolving, always changing. I think the key word that comes up quite a lot is adaptability. And I think that you’re nailing that at the moment. So moving away from the smart home then. So in a competitive market that face at the moment, how can businesses accelerate their IoT deployments? Using test solutions like your Passport?

Tim Winters
Yeah, so what you know, I think what’s happening there is the, you know, I hit on this earlier, but I, the automation, automation, being able to run it, repeat it, you know, there’s nothing worse than having a bug and not being able to reproduce it. And then some poor developer is trying to figure out what scenario that was in to cause that issue. So what we’re trying to do there is to make it more repeatable easy, so that these devices, you can drill down, find the issue, the developer can make a fix, see if it broke anything, because then you’ve got the regression issue, all of that we’re trying to make it repeatable, automated part of your regular process, right? What you don’t want is it to be at the very end, because you’re either done with that project, and the developers moved on to something else, right. And it’s going to be hard to get them to put their eyes back on the individual project to fix it.

Tim Winters
So it’s really trying to make it part of the whole process so that you can address issues as they happen, you know, and as companies deploy IoT things, it’s a tricky market, because you’re not sure how long it’s going to be deployed, depending on what it is. There are things IoT is a little both ways, which has been fascinating, right, is that there are devices that aren’t long, they know they’re not going to be out there long, they’re gonna get replaced really quickly. And then there’s other IoT devices, you know, think of appliances in your house that you don’t often replace, like, think of boiler or something like that, right? You’re not replacing your boiler every three years, at least I hope not. Right. And so that’s good. That’s where this tug and pull where those guys need to do more testing or need to be more prepared, because their device is going to live on for a long a while. And so it’s kind of fascinating. We’re trying to help people figure that out. Because there’s definitely testing levels, as I will say, there are customers who we need this to work, we need this to be able to update and be connected all the time, it would be really bad if it’s not. And then we have other customers who are definitely looking for major issues, because the lifetime of it isn’t, isn’t as long. And so it’s been kind of fascinating working through that those types of environments.

Brad King-Taylor
Yeah, this is why I think this is something that can be cross industry, whether it’s businesses or homes. You’ve touched on it, the testing could be implemented a lot of different things. One thing I personally hate is having to replace something that I’ve already got. There is nothing more annoying, whether it’s a totally different because if I have to replace a telly, I’ll just go out another size. But washing machines or anything like that, yes, nothing more annoying. Yeah, that’s true, right? Yeah. If I’ve already got it. I don’t need another one.

Tim Winters
Yeah, the tough one for me was the light bulbs having to replace the light bulbs in my house. I couldn’t actually do it. I had to wait for them to burn out because I got a bunch. And then I was like, but I was like, No, I couldn’t. I couldn’t just but now I have, you know, over half my house has the smart light bulbs that, you know, you can walk into a room and tell them what to do. But I’m totally with you. I couldn’t just replace them all with all these. You know, all I couldn’t do it all at once. Just couldn’t stomach it.

Brad King-Taylor
Put your pain barriers. They’re wonderful. Yeah. Which is money. Right? Yeah. Right. So moving on for Tim. So where do you see the future? In terms of the smart home devices, the IoT smart devices in the future? What’s going on there? Is there going to be more innovation? When it comes to testing? Are we expected to see more from QA cafe for instance?

Tim Winters
Yeah, I definitely think we’re gonna, you know, evolve with the industry, I do think that we’re seeing some you no matter the chip group is a bunch of Amazon, Google and Apple getting together trying to make more interrupt, I do think people understand that this is not going to be a proprietary setup. And it’s going to continue to be diverse. And so we’re going to be there for all of that, we’re going to be there to help test, work with these customers to make the networking better, because that’s what we want. We want people to have a good experience. That’s what all of these companies or networks, you know, the network providers, all what they’re trying to do is make the network better. I think as it evolves going forward, obviously, we hit on security earlier, that’s going to continue to be an evolution. I also think you know, there’s going to be some really interesting stuff going forward about how these devices interact with each other in a variety of different ways.

Tim Winters
So I expect all of that to continue going forward in that space. And we’re going to we’re going to continue to work try to automate this and make it as easy as possible for customers to be able to develop things that will be go out in the network and do useful things. So They can focus on that, instead of the weird networking problem they might have, because that’s one of the things that we’ve noticed too, is some companies are very good at building a product. But networking is not their thing. And so we’re hoping to help them build devices that they don’t have to worry about the networking side, and they can focus on making sure that your your grill temperature, your thermometer and your grill is giving you the right thing and not that it can connect to your network.

Brad King-Taylor
Because if you’ve already got a grill, you don’t want to replace it. I mean, one thing that does excite me is you’ve touched on it is how companies or devices can start to link together. In terms of Apple, Google, it’d be really good if they all work together. I’m a little bit sceptical, personally on how long that’s gonna take. But hopefully it does happen, because it’ll be amazing. Yeah, brilliant. I think the whole the whole mantra and ethos of what you’re trying to achieve is fantastic. And I can see this growing something quite big. As always, at the end of our podcast, we don’t just finish with our questions and what we planned. It’s, it’s overlapping. So there is a question from the previous podcast, which is Roy Dagan CEO and co founder of Securithings. And his question to you as customers are using IoT for hundreds of different use cases. So you must have seen a very large variety, has there been more interesting stuff to work with? Or that you’ve seen?

Tim Winters
Yeah, absolutely. So you know, the interesting things we’ve seen is we’ve had a lot of customers come to us, probably the biggest thing cuz people came to us with was all the different failover test cases. So it’s working fine. Something happens in the network. And it’s not directly connected to the device, some service far off dies, and it doesn’t come, the device doesn’t come back, right. And it just hangs out. And it’s doing nothing. And that was one of the biggest things. So we created a bunch of scenarios that happened in networks to make sure that those devices can recover. I would say device recovery is probably the most interesting scenarios we’ve been given from ISPs because things change, they want to move things around in their network. And then these IoT devices become very pretty pieces of, of brick. So that’s one of the things we’re helping work on. For sure. I think that’s probably the most interesting cases we’ve seen.

Brad King-Taylor
Perfect. Thank you. I’m sure that was an answer that he was open to hear. look amazing, amazing podcast. Thank you so much. And I wish you all the best for the future wish everything goes well. Yeah, thank you. Thank you very much. The only that the only last thing to do is to touch on your socials, your websites and anything. Where can the listeners find QA cafe? Yeah,

Tim Winters
Yeah, you can find us you can find this in QA cafe. So www.qacafe.com. And that has all of our information and our social stuff is there. We’re on Twitter and Facebook and all of the all of the wonderful social platforms, you can find this.

Brad King-Taylor
Perfect. Tim, pleasure. Thank you so much. Thanks. Please don’t forget to make sure you leave a like and subscribe to the podcast on whatever platform you’re listening or watching on. It gives my team the ability to give you all the content you love, as always, doesn’t matter how you’re connected as long as you’re connected. Until next time. We’ll see you soon.

The IoT Podcast Team

The IoT Podcast is powered by Paratus People, a leading organisation in IoT Talent Solutions.

Innovation is at the heart of IoT, it is our passion to explore and learn more about this fast paced and transforming sector.

Connect & Get Involved

Your subscription could not be saved. Please try again.
Your subscription has been successful.
Subscribe to our newsletter to be amongst the first to find out exclusive information about The IoT Podcast.

We use Sendinblue as our marketing platform. By Clicking below to submit this form, you acknowledge that the information you provided will be transferred to Sendinblue for processing in accordance with their href="https://www.sendinblue.com/legal/termsofuse/">terms of use