Introduction

Fifteen years ago, the average household only had a handful of wireless devices: a work laptop, one or two smartphones, a smart thermostat, and, perhaps, a smart speaker. Then, streaming video services such as Netflix and video conferencing like Zoom and Google Meets ramped up in popularity, with the pandemic accelerating their usage with work-from-home becoming more common. To meet those demands, mesh WiFi systems alleviated the bottlenecks of a single AP, with distributed coverage around a house, and CSPs turned to managed WiFi systems to gain visibility into the operations of the homes of their customers.

More recently, it is difficult to buy a “non-smart” appliance without WiFi connectivity. Whether it’s a refrigerator, washing machine, or television, device manufacturers see the value of incorporating connectivity into their devices to troubleshoot, track usage, and offer new features to their customers. Additionally, IOT devices such as IP video cameras, doorbell cameras, smart front door locks, and smart lighting are becoming more popular as DIY solutions for smart automation. Finally, personal devices such as smartwatches and bluetooth audio devices have become ubiquitous, further demanding time in the shared airwaves.

This increasing number of wireless devices presents challenges and opportunities. The challenges include:

Providing reliable, high bandwidth connectivity to such a large number of devices in the home
Adapting the network to individual homes, and dynamic loads from a broad set of devices with diverse networking requirements
Managing and monitoring a large range and number of devices
Ensuring security, made difficult by the number of devices, importance of the devices in consumers every day activities, and the accelerating frequency and sophistication of attacks

However, this revolution in home electronics brings opportunities as well:

Devices distributed across the home that can act as sensors
A large amount of under-utilized compute power within the home
Consumer familiarity and desire for high levels of automation and sophisticated functions

More specifically, the types of new applications/services that can be built on the distributed home electronics network include cyber and physical security, content protection and parental controls, occupancy sensing and control, image processing, and sensing and home automation systems.

Fortunately, recent advances in artificial intelligence (AI) have been equally as dramatic. AI has already revolutionized everything from internet search, to image processing, to generative tasks such as writing text, to physical applications such as self-driving cars. It turns out that AI is also the perfect tool to meet the challenges and seize the opportunities of an electronics filled modern home.

While AI is available as a tool, for it to be effective in the home the correct underlying architecture needs to be created together with the appropriate applications. RighValor was developed exactly for this purpose.

Architecture

Figure 1 shows a top level diagram of the ideal architecture, together with an appropriate base set of applications in the home. The architecture can be thought of as four layers. The bottom layer holds the underlying network connectivity layer, known as RighGravity. Above that is an AI infrastructure layer, followed by a flexible and extensible set of AI powered applications, which together are known as RighValor. At the top is the cloud layer, which performs federated learning, and the cloud portion of any applications. In the figure, two nodes/devices are shown, although the architecture can support an indefinite number of devices in the home.

While the focus of this paper is RighValor, a quick description of RighGravity and how it relates to RighValor will be helpful.

RighGravity

There are a wide variety of WiFi management systems available today. Many CSPs, equipment vendors, and even chipset vendors have already chosen one or more of these systems to support. However, none of these systems have all the required features, and some have significant drawbacks in cost and robustness. However, it is impractical for a CSP to replace all the hardware they already have in the field, and risky and disruptive to try to replace all their current software in the field and in the cloud.

RighGravity was designed as the first WiFi management system to work in conjunction with existing management systems, augmenting their capabilities to complete the required feature set. In order to accomplish this, RighGravity has exceptional modularity, clean abstraction layers, strict decoupling of the control and data planes, and unparalleled flexibility. Matched with Righ’s unparalleled porting efficiency and experience, RighGravity is the most easily deployed WiFi management system in the industry.

Beyond ease of deployment, RighGravity has a collection of capabilities that meets the challenges of today’s home networks. Figure 2 lists the capabilities. A key aspect of RighGravity is its clever use of cloud resources. It can operate completely independently, as a standalone locally controlled network, or it can take advantage of cloud resources for AI features. In either case, since it leverages local compute, it has superior latency, robustness, and cost to systems that are highly cloud dependent. As will be described, RighValor similarly makes clever use of local and cloud resources.

RighGravity creates the perfect foundation for RighValor. The following features enable the required AI infrastructure and AI based applications to meet the challenges and achieve the opportunities listed earlier:

Reliable, high bandwidth communication between all devices in the home
An extensible software platform, that can support the addition of AI processing
The ability to collect data from local sensors
Movement of data among local and cloud compute and storage resources

More details about RighGravity can be found in its white paper. With the RighGravity foundation in place, RighValor can leverage the sensors and compute resources in the local devices to provide a wide range of new applications.

RighValor Infrastructure

RighValor layers an AI infrastructure on top of RighGravity. This infrastructure uses primarily local processing for its AI functions, with support from the cloud to enrich the AI models. To accomplish this, RighValor includes the following capabilities:

Elastic Edge Computing: AI processing jobs can be run locally on any device that has RighGravity, including gateways, extenders, set top boxes, even mobile phones and laptops. RighValor includes clustering and resource virtualization across multiple nodes. This is similar to how technologies like Kubernetes have been used in cloud development. It is novel to apply this approach not across cloud servers, but across local devices with compute capability. In addition to distributing multiple tasks, a single AI processing task can be parallelized across multiple devices, distributing the load and accelerating the completion of the computing task.

Orchestration: The AI processing jobs are coordinated by an orchestrator, which is part of the Linux Container Daemon described later. The orchestrator assigns jobs to each computing node, accommodating other compute requirements on each node (such as network traffic processing) and balancing the load across idle or lightly loaded resources. This allows the compute power to scale with demand, and inherently creates fault tolerance in the system.

Federated Learning: While RighValor does most of its AI processing locally, including inferencing and training of models, federated learning is applied in the cloud to leverage learning gained in one home to all homes. Key to this is the ability to pass learned parameters and select attributes and examples to the cloud for analysis. This provides a large number of advantages:

The processing required in the cloud is minimized (and operation with no cloud at all can be supported)
Latency for time critical tasks is reduced
Nearly all data is kept local, reducing privacy concerns
Learning is naturally specific to each individual home
However, learning is improved and accelerated by sharing insights between homes
Very complex tasks, such as development of a new AI method, can utilize the full sophistication of cloud-based AI methods with real examples from a large sample of homes

Containerized Deployment: By utilizing Linux Containers (LXCs) the AI software can be made easily portable to different platforms. LXCs provide a lightweight, secure, and efficient way to run applications in isolated environments that provide consistency of operation. Along with ease of deployment and updating, Righ’s container system ensures scalability, maintainability, and secure execution. Righ even includes a LXC Daemon that serves as a higher-level container manager, providing an easy-to-use interface for container creation, management, and automation. It will manage the lifecycle of containers, including provisioning, scaling, and destroying containers.

With RighGravity forming the underlying network, and RighValor infrastructure creating an ideal environment for AI in the home, a wide variety of AI based applications can be deployed into the home.

RighValor Applications

There are more “sensors” in a home than at first glance, if you realize that such things are network use data, WiFi channel estimates (which reflect motion in the home), and data from cameras are all useful sensor data, each supporting a variety of applications. Based on the data from these sensors, there are a huge number of different applications that can be implemented on networked devices in the home and enhanced with AI. Just the initial applications are described in this paper.

WiFi Management

Older legacy networks may struggle with twenty or thirty devices trying to access the network during peak hours. But, modern-day (WiFi 6 and 7) tri-band networks generally have the capacity to handle these loads if properly configured. This means using frequency channel diversity to spread traffic to avoid congestion, moving higher bandwidth devices to the “super-highways” so that the slower traffic does not clog up the arteries, and prioritizing more critical devices. In practice, this is easier said than done, since each network is unique. One location may have a single adult that works and watches tv from the same room, while a second may have two adults and two kids that work in multiple rooms at the same time.

AI can be applied to learn the patterns of use in each individual home, allowing the network to be configured so as to optimize the network for the loads and devices present. RighValor provides such optimization, utilizing local processing and federated cloud based learning to combine local conditions with home specific conditions while minimizing cloud processing, storage, and transport costs.

Device Classification

Device classification refers to identifying the type of device as well as the likely user of the device. Device classification is a particularly challenging problem since WiFi devices do not identify what type of device they are as part of the WiFi protocol. In fact, many WiFi devices now disguise themselves each time they connect by scrambling what used to be a fixed field, their MAC address. However, device classification can be very helpful for a variety of tasks:

Optimizing the WiFi network, including steering devices, by identifying those devices likely to require a high data rate stream vs. those that have modest service requirements
Showing users the topology of their network, including whose and what type of device is connected where in the network
Applying content filtering to protect children from malicious or inappropriate content

Conventional approaches to content filtering require the users to manage everything themselves, setting sites to block or allow, and identifying which devices are in which set of block/allow categories. In fact, many systems require the user to install software on each individual device, which is at best inconvenient, and impossible for some classes of devices that children use.

Figure 3 shows RighValor’s system for doing device classification system. It applies an AI model to determine both the type of device and the likely age of the user based on observing the network traffic that the device transmits and receives. RighGravity is able to provide the input data to this model, which runs in the RighValor containerized infrastructure system. RighGravity further allows the system to take corrective action, blocking the traffic carrying inappropriate content. It can even automatically block different types of traffic at different times of the day, enforcing better pre and post bedtime habits. The training required for this AI model is not extensive. As an example, Righ was able to achieve 87% accuracy with a limited training set of just 10 households consisting of 27 adults and 10 children.

Figure 3 Righ Device Classification System

RighValor’s approach to this problem provides several advantages. The federated learning approach keeps most of the data locally, particularly the sensitive browsing history, ensuring privacy. Local learning allows the model to evolve over time. For example, if a block is overridden (white listed), the AI engine can learn this and adjust the threshold for similar websites. The federation connection to the cloud, along with containerization, allows the cloud to provide learning assistance (for example a starting point for rules when the network is first installed), and even change the AI methods over time.

Occupancy Monitoring

Technology has been developed that uses the WiFi signals naturally crisscrossing a home to detect movement in the home. This has been enhanced to the level where even breathing can be detected under laboratory conditions. However, the technology has proven difficult to commercialize due to the variety of conditions in each household. By applying AI processing with local learning, the model can be tuned for each home individually, adapting to specifics of the location, devices, people, and even pets. As with the other applications, the combination of local AI processing and cloud federated learning protects privacy, allows the use of large amounts of data and processing at low cost, while providing for sophisticated improvement over time.

Network traffic itself can also be observed to aid in this detection. RighGravity can provide both the change in the radio propagation channels and network traffic observations to the occupancy AI model, which can be trained to recognize local patterns.

Sensor Networks

The typical home could benefit from a significant number of sensors. These would include window and door sensors for security, temperature sensors throughout the home for heating and air conditioning, light sensors to adjust lighting and shades, and a variety of cameras for doorbells and security. Getting data or power wiring to all these locations is impractical. The best solution is for the sensors to be wirelessly connected and battery powered. Wireless technology has evolved to the point that the energy required to transmit a bit of information is similar to the energy required to write that bit of information to local memory (~100nJ/bit in both cases). This can make offloading the sensor data and computation to an AC powered device in the home attractive from a power perspective.

Of course, the data could be moved all the way to the cloud for processing. But this carries the disadvantages of longer latency, loss of privacy, and ongoing cloud costs. The ongoing costs are particularly problematic, as for some types of devices, consumers may be willing to make a one-time purchase, but are unwilling to pay an ongoing subscription fee. In fact, local processing allows more extensive and continuous processing since costs don’t need to be minimized.

Image Processing

Image processing is really just a particular type of sensor, but it is so important it deserves its own category. A classic use case for cameras is in an alarm system, in which case scene detection (what is going on), and person identification (who is doing it) are critical. Figure 4 shows the implementation of a vision recognition system in which a remote battery powered camera sends its data to the AP in the home, which then uses AI to perform scene and person recognition. The AI system includes federated cloud support to enhance local learning with insights from other locations.

The computational requirements for the image processing are well within the capabilities of typical devices in the home such as a WiFi APs. For example, person detection can be done at 1.2fps, based on a memory footprint of 23MB, using 106MB of RAM, while consuming 83% of a single CPU. Given that typically APs have four or more CPU cores, this is a manageable load even for low end APs.

Conclusion

Breakthroughs in AI have made an entirely new set of applications possible for the home. Righ has architected and implemented a complete system that provides the necessary infrastructure as well as an initial set of applications. RighGravity forms a solid networking layer between nodes in the home and the cloud. In addition, it provides a platform on which AI processing software can be added, while gathering and exchanging data from the networking system and sensors in the devices. RighValor includes an AI infrastructure layer, which includes elastic edge computing, orchestration of processing distributed across local devices, federated learning with the cloud, all delivered as easily maintained and secure containers. RighValor also includes a foundational set of applications including WiFi management, device classification, parental controls, occupancy monitoring, image processing, and support for sensor networks. Righ’s unique architecture results in a system that has the advantages of home specific learning, enhanced privacy, reduced latency, and significantly lower cost.

Contact Righ

RighValor: AI Infrastructure and Applications for the Home