BitLaw

Example 36: Tracking Inventory

The following fact pattern and claims are hypothetical. Assume that the claims are presented in a recently filed application that is under examination and thus each claim is given its broadest reasonable interpretation in view of the specification as it would be understood by one of ordinary skill in the art. In this example, the terms in the claim are given their plain meaning in the art because no special definitions have been set forth in the specification. An abbreviated version of the hypothetical specification is provided below. Claim 1 is ineligible, because it is directed to an abstract idea and does not recite additional elements that amount to significantly more. Claims 2 and 3 are directed to the same abstract idea, but are eligible because they recite specific limitations other than what would be well‐understood, routine, conventional activities in the field, which amount to significantly more (i.e., provide an inventive concept).

Background

Inventory management is a commercial practice involving the acquisition and monitoring of stocked goods to maintain stock levels in a business. Particularly when goods are stored in large warehouses, managing inventory requires monitoring what goods are currently in stock and where those goods are located in the warehouse in order to fulfill orders in an efficient manner. Some prior methods of tracking inventory required items of inventory to have an attached tracking device such as a RFID or GPS transmitter, but these methods were cumbersome to implement since each item needed a transmitter to be affixed and detached as the item entered and exited the warehouse. In addition, these methods could not accurately track an item if the transmitter was obscured, improperly affixed, or detached from the item. Other prior methods used imaging technology to acquire and process images to track the items of inventory, but these methods did not have much success because they used the view of a single camera to track an object and attempted to identify items solely based upon character data (such as identification codes or product names) printed on the item. Due to using the view of a single camera to track an object, it was difficult to determine an object’s physical three‐ dimensional (3‐D) location. Therefore, these methods required items that were moved to be reimaged or otherwise tracked through manual scanning or logging. Mistakes in data entry or failure to scan a moved item resulted in lost or misplaced items. Accordingly, previous attempts to implement image recognition to track items of inventory have not achieved a high rate of accuracy.

Applicant has invented a system for tracking the presence and location of items of inventory in a warehouse using an integrated camera system with computer vision technology that overcomes many of the problems in the existing technologies typically used in the industry. Applicant’s system overcomes the issues relating to accurately identifying items and tracking missing items by using a high resolution video camera array with overlapping views in combination with a recognition model that uses not only the character data of the item but also contour information (i.e. shape) from the collected images and predictive location data. By using a combination of character and contour recognition, applicant’s system greatly reduces the possibility of item misidentification and significantly improves accuracy of inventory over prior techniques that used only character information. Because the cameras in the array have overlapping views, objects can be tracked across multiple cameras and the 3‐D location of the objects can be automatically reconstructed. Applicant’s improvement to computer vision technology to manage inventory within existing warehouse operations thus results in more accurate inventory tracking while eliminating the need for procedures such as scanning and logging items.

In practice, the invention uses high resolution video cameras positioned to have overlapping fields of view in pre‐determined locations throughout the inventory storage space. Such cameras enable the system to automatically track an item across the entire storage space and estimate its physical location. An inventory recognition model is also stored in the memory and comprises a mathematical representation of each item of inventory handled by the particular warehouse. This model may be a Gaussian mixture model, neural network, Bayes classifier or other known pattern classifier. The model is developed using a supervised training algorithm using numerous images of each item at multiple distances and positions with respect to the camera. During training, characteristics of each item are extracted from the images including character information such as the item’s name and identification code and contour information such as the shape of the item and/or the shape of the packaging for the item. The recognition model may be updated as needed when items are added or discontinued.

During operation, the video cameras capture an image sequence (e.g., multiple images from one or more of the cameras) comprising overlapping images of an item, which is stored in the memory in an inventory record. The system then uses a programmed computer to extract characteristics of an item including character and contour information from the high resolution images in the image sequence using a combination of existing text and edge detection algorithms. The programmed computer uses the characteristics to form feature vectors, and then classify the item by processing the feature vectors with the inventory recognition model to determine the most likely item in the image. A positive recognition result indicates the presence of the item in the warehouse. After an item is recognized, it is tracked in real‐time throughout the warehouse using a tracking algorithm that takes advantage of the overlapping camera views to confirm the location of the item (thus improving retrieval time and accuracy). Specifically, the item is tracked in the image sequence of one camera using a known method, such as Kalman filtering, and once that item enters the field of vision of a second camera, its position in the first camera’s view is used to quickly locate the item in the second camera’s view. The item can then be tracked similarly in the image sequence of the second and subsequent cameras. The computer then reconstructs the 3‐D coordinates of the item based upon the item’s location in multiple overlapping images and prior knowledge of the location and field of view of the camera(s) that are tracking the item. Finally, the computer updates the item’s inventory record with the 3‐D location information.

In this hypothetical scenario, computer vision technology has not been used in the manner disclosed by this inventor prior to the filing of the application.

Claims

1. A system for managing an inventory record comprising a memory and processor configured to perform the steps of:

   (a) creating an inventory record for an item of inventory comprising acquired images of the item;
   (b) adding classification data relating to the acquired images to the inventory record;
   (c) adding location data relating to each acquired image to the inventory record; and
   (d) updating the inventory record with a physical location of each item of inventory in the warehouse to thereby manage the items of inventory.

2. A system for managing an inventory record by tracking the location of items of inventory in a warehouse:

   a high‐resolution video camera array, each video camera positioned at pre‐determined locations with overlapping views, for acquiring at least one high‐resolution image sequence of each item of inventory;

   a memory and processor configured to perform the steps of:

   (a) creating an inventory record for an item of inventory comprising the acquired image sequence of the item from the video camera array;
   (b) adding classification data relating to the acquired image sequence to the inventory record;
   (c) adding location data relating to each acquired image to the inventory record, the location data providing a position of the item of inventory in the image sequence;
   (d) reconstructing the 3‐D coordinates of an item of inventory using the location data from multiple overlapping images and prior knowledge of the location and field of view of the camera(s); and
   (e) automatically updating the inventory record with the 3‐D coordinates of each item of inventory in the warehouse to thereby manage the items of inventory.

3. A system for managing inventory by tracking the location of items of inventory in a warehouse using image recognition, comprising:

   a high‐resolution video camera array for acquiring at least one high resolution image sequence of each item;

   a memory for storing the acquired image sequences, classification and location data relating to the acquired image sequences, and a recognition model representing contour information and character information of each item; and

   a processor that is configured to manage inventory by performing, for each item, the steps of:

   (a) creating an inventory record for the item comprising the acquired image sequence(s) of the item;
   (b) extracting characteristics from the acquired image sequence(s) of an item to form feature vectors, the characteristics comprising contour information and character information that is stored in the inventory record as classification data relating to the acquired image sequence(s);
   (c) recognizing and tracking the position of item in the image sequence as classification and location data by processing the feature vectors using the stored recognition model and adding the classification and location data to the inventory record;
   (d) determining a physical location of the item in the warehouse using the location data relating to the item in the image sequence(s); and
   (e) automatically updating the inventory record with the physical location of the item.

Analysis

Claim 1: Ineligible

The claim recites a system for managing an inventory record comprising a memory and a processor configured to perform a series of steps. The claimed system is a device or set of devices, which is a machine and thus a statutory category of invention (Step 1: Yes).

The claim is then analyzed to determine if the claim is directed to a judicial exception. The claim recites a system that performs the steps of (a)‐(c) storing acquired images and related classification and location data, and (d) updating the inventory record with the physical location of each item of inventory in the warehouse. That is, the claim describes the steps of managing inventory by creating an inventory record for each item of inventory comprising images of the item, adding classification data relating to the images to the inventory record, adding location data for each image to the inventory record, and updating the inventory record with the physical location of each item of inventory in the warehouse. The data collection, recognition, and storage concept described in the claim is similar to the data collection and management concepts that were held to be abstract ideas in Content Extraction, TLI Communications, and Electric Power Group. Although the claim enumerates the type of information (i.e., the images, classification data, and location data) that is acquired, stored and analyzed, the Federal Circuit has explained in Electric Power Group and Digitech that the mere selection and manipulation of particular information by itself does not make an abstract concept any less abstract. Further, the claim is not made any less abstract by the invocation of a programmed computer. Unlike Enfish, where the claims were focused on a specific improvement in how the computer functioned, the claim here merely uses the computer as a tool to perform the abstract concepts. Therefore, based on the similarity of the concept described in this claim to abstract ideas identified by the courts, claim 1 is directed to an abstract idea (Step 2A: Yes).

Next, the claim as a whole is analyzed to determine whether any element, or combination of elements, is sufficient to ensure the claim amounts to significantly more than the abstract idea. The claim recites the additional limitations of a memory and processor to perform the steps of inventory tracking. A memory for storing data and a processor for processing data are well‐understood, routine, conventional computer components, which in this claim are recited at a high level of generality and perform generic computer functions (e.g., storing and processing information). Generic computer components performing generic computer functions, alone, do not amount to significantly more than the abstract idea.

Viewing the limitations in combination also fails to amount to significantly more than the abstract idea. The claimed invention seeks to record, process, and archive digital images simply, fast, and in such a way that the information may be easily tracked, but these functions reflect ordinary usage typically performed by a generic computer, as would be recognized by those of ordinary skill in the field of data processing. For example, as noted in TLI Communications, using a computer to attach classification data, such as dates and times, to images for purposes of storing those images in an organized manner does not add significantly more to a judicial exception. The recitation of conventional processing technology performing well‐understood, routine, conventional functions such as recognizing and storing data from specific data fields does not reflect an “inventive concept.” Thus, whether viewed individually or in combination, the additional limitations do not amount to a claim as a whole that is significantly more than the abstract idea (Step 2B: No). The claim is not patent eligible.

A rejection of claim 1 should identify the abstract idea by pointing to the language of the claim that describes inventory management and explaining that inventory management is similar to concepts that courts have previously found abstract. The rejection should identify the additional limitations regarding the memory and processor and explain why those limitations comprise only a generic computer performing well‐understood, routine, conventional generic functions in the particular technological environment of image processing, for the reasons noted above.

Claim 2: Eligible

The claim recites a system comprising a video camera array, a memory and a processor. The system is a device or set of devices and therefore is a machine, which is a statutory category of invention (Step 1: Yes).

The claim is then analyzed to determine if the claim is directed to a judicial exception. Like claim 1, claim 2 recites a system that performs the steps of (a)‐(c) acquiring and storing images and related data about items of inventory, and (e) updating the inventory record with the physical location of each item of inventory in the warehouse. Claim 2 thus describes using data collection and management techniques to practice the concept of inventory management, which as explained above is an abstract idea. Therefore, the claim is directed to an abstract idea (Step 2A: Yes).

Next, the claim as a whole is analyzed to determine whether any element, or combination of elements, is sufficient to ensure the claim amounts to significantly more than the abstract idea. The claim recites the additional limitations of a high‐resolution video camera array at predetermined positions with overlapping views, memory and processor to (d) reconstruct the 3‐D coordinates of the item of inventory from multiple overlapping images obtained from the camera array and prior knowledge of the location and field of view of the camera(s). Individually, the memory and processor limitations do not amount to significantly more for the reasons discussed above for claim 1. For example, they are still well‐understood, routine, conventional devices that are used in this invention for their conventional functions of processing and storing information. Similarly, high‐resolution video cameras are widely used and, in this invention, perform their typical function of acquiring image sequences.

However, the memory and processor in combination with a high‐resolution video camera array with predetermined overlapping views that reconstructs the 3‐D coordinates of the item of inventory using overlapping images of the item and prior knowledge of the location and field of view of the camera(s) provides significantly more than the abstract idea of using data collection techniques to manage inventory. As explained in the specification, at the time of this invention, using a high‐ resolution video camera array with overlapping views to track items of inventory was not well‐ understood, routine, conventional activity to those in the field of inventory control. In fact, the use of this camera array provides the ability to track objects throughout the entire storage space rather than simply the view of a single camera and determine their 3‐D location without any of the manual steps that were required of previous methods. That is, the video camera array with reconstruction software provides the technological solution to the technological problem of automatically tracking objects and determining their physical position using a computer vision system. Like in DDR, the claimed solution here is necessarily rooted in computer technology to address a problem specifically arising in the realm of computer vision systems. The claimed limitations are not simply an attempt to generally link the abstract idea to the technological environment of computer vision systems. Rather, these are meaningful limitations that confine the claim to a particular useful application. Accordingly, when viewed as a combination, the additional elements thus yield a claim as a whole that amounts to significantly more than the abstract idea of inventory management (Step 2B: Yes). The claim is patent eligible.

If the examiner believes the record would benefit from clarification, remarks could be added to the Office action or reasons for allowance indicating that the claim recites the abstract idea of inventory management. Nevertheless, the claim is eligible because analyzing the claim elements in combination demonstrates the claim is a technology‐based solution to address a problem arising in the realm of computer vision systems and is not simply limiting the abstract idea to a particular technological environment.

Claim 3: Eligible

The claim recites a system comprising one or more video cameras, memory and a processor. The system is a device or set of devices and therefore is a machine, which is a statutory category of invention (Step 1: Yes).

The claim is then analyzed to determine if the claim is directed to a judicial exception. Like claim 1, claim 3 recites a system that performs the steps of (a) & (c) storing acquired images and related classification and location data, and (e) updating the inventory record with the physical location of each item of inventory in the warehouse. Claim 3 thus describes using data collection and management techniques to practice the concept of inventory management, which as explained above is an abstract idea. Therefore, the claim is directed to an abstract idea (Step 2A: Yes).

Next, the claim as a whole is analyzed to determine whether any element, or combination of elements, is sufficient to ensure the claim amounts to significantly more than the abstract idea. The claim recites the additional limitations of a high‐resolution video camera array for acquiring high resolution image sequences of items of inventory, a memory to store the acquired images, related data, and the recognition model, and a processor to perform step (b)’s extracting characteristics from the acquired images, step (c)’s recognizing and tracking the position of the item using the recognition model and step (d)’s determining a physical location of the item using the position of the item in the images. Individually, the camera array, memory and processor limitations do not amount to significantly more for the reasons discussed above for claims 1 and 2. For example, these components are used in this invention for their well‐understood, routine, conventional functions of acquiring, processing and storing information.

In combination, however, the limitations do amount to significantly more than the abstract idea of inventory management. As explained in the specification, the combination of the camera array’s acquisition of high resolution image sequences, and the processor’s performance of step (b)’s extracting contour and character information from the images to create feature vectors, step (c)’s recognizing and tracking items of inventory using the feature vectors and a recognition model, and step (d)’s determining the physical location of the recognized items using the position of the item in the image sequence(s) is not well‐understood, routine, conventional activity in this field. This combination of limitations provides a hardware and software solution that improves upon previous inventory management techniques by avoiding the cumbersome use of RFID and GPS transmitters and the inaccuracy issues that plagued previous computer vision solutions. This combination of features provide meaningful limitations to the practical application of inventory tracking with computer vision, by improving the system’s ability to identify and track objects across multiple cameras in a three‐dimensional space. These limitations do not simply limit the abstract idea to the technological environment of image processing, but are instead meaningful limitations that integrate the abstract idea into a particular application that uses character and contour information from high resolution images to recognize items of inventory. When viewed as a combination, the additional elements thus yield a claim as a whole that amounts to significantly more than the abstract idea of inventory management (Step 2B: Yes). The claim is patent eligible.

If the examiner believes the record would benefit from clarification, remarks could be added to the Office action or reasons for allowance indicating that the claim recites the abstract idea of inventory management. Nevertheless, the claim is eligible because analyzing the claim elements in combination demonstrates the claim is a particular application rather than well‐understood, routine, conventional activity or simply limiting the abstract idea to a particular technological environment.