Skip to content
  • Our Work
    • Fields
      • Cardiology
      • ENT
      • Gastro
      • Orthopedics
      • Ophthalmology
      • Pulmonology
      • Surgical Intelligence
      • Surgical Robotics
      • Urology
      • Other
    • Modalities
      • Endoscopy
      • Medical Segmentation
      • Microscopy
      • Ultrasound
  • Success Stories
  • Insights
    • Magazine
    • Upcoming Events
    • Webinars
    • Meetups
    • News
    • Blog
  • The company
    • About us
    • Careers
Menu
  • Our Work
    • Fields
      • Cardiology
      • ENT
      • Gastro
      • Orthopedics
      • Ophthalmology
      • Pulmonology
      • Surgical Intelligence
      • Surgical Robotics
      • Urology
      • Other
    • Modalities
      • Endoscopy
      • Medical Segmentation
      • Microscopy
      • Ultrasound
  • Success Stories
  • Insights
    • Magazine
    • Upcoming Events
    • Webinars
    • Meetups
    • News
    • Blog
  • The company
    • About us
    • Careers
Contact

Temporal point process sampling in video

Object identification and tracking in a sequence of frames (video) consists of sampling of the scene, by e.g raster or uniform scatter, to extract features and compute their descriptors for target objects identification. This raster scanning procedure can by resource intensive, especially if every (or almost every) pixel in the image needs to be examined; hence this poses as a bottle neck for real time applications. Although in sparse image application only points of interest (such as edges) are to be examined, full image operations are still utilized (gradients or extracting corners), which requires information from all pixels. Heavily crowded scenes, such as urban landscape, containing pedestrians, cars and traffic signal, need to be identified and tracked by an automated driving assistance system (ADAS) and integrated, to be able to issue a timely response. However, for some applications, sampling space can be reduced if the temporal link between the expected successive locations of objects is retrieved.

.

Learn to search for information in designated areas
Samples in green are illustrations showing a possibility where the algorithm searches for information in each frame (say from a dash cam)

.

The goal, therefore, is to dramatically reduce the image sample by predicting which pixels to sample on which we compute features and descriptors for further analysis, identification and tracking. The mathematical tool which deals with inference of the conditional (temporal) link between successive distributions of events (points related to target object at any time) is called temporal point processes. Temporal point processes are stochastic realizations of points scattered in space at time ti, having an intensity (loosely, the expected number of points per unit measure) conditioned on the intensity (or distribution) in the previous time step ti-1. The distribution of points is unknown a priori and needs to be estimated from the data.

.

The algorithm searches for information in each frame
Rather than scanning the whole image, the machine learning algorithm learns to search for information in designated areas

.

Several types of temporal point processes are useful for object points (events) identification. For example, the self-excitatory process (Hawkes process, which is also used to model location of residual secondary earthquakes after a large shock) increases the sampling of points near a successful event (identification of object point) in subsequent time steps, thus providing more “relevant” samples in the vicinity of the object to be extracted. On the other hand, self-inhibitory processes diminish the number of points in the vicinity of “failed” or irrelevant events (non-object sample point), thus reducing the sample space where it is not needed.
 .
We therefore arrive at the crucial point: how to retrieve such dynamic sampling based on temporal point processes. Such non trivial relationship must be learned and adapted dynamically from the data. We find the solution in Machine Learning methodologies, the goal of which is to continuously update the conditional intensity of the point process based on its history. The parameters of the conditional intensity of our point processes (or mixed inhibitory-excitatory process) can be obtained by minimizing log likelihood, thus obtaining the maximum likelihood estimators.
 .
In the learning phase, we need to feed our network (e.g. recurrent neural network) with identified object points in each frame. Special attention must be given to applications for which the view point is static (e.g. security cameras) vs. dynamic (e.g. dash cameras in cars). These cases force the restriction that sampling process should converge to either the homogeneous Poisson point process (constant intensity) in the case of no temporal correlation, or to the excitatory-inhibitory mixed distribution (non-homogeneous), when object identification has some certainty. The use of cutting edge methodologies in computer vision and machine learning has been the everyday practice at RSIP Vision for years. Our engineers develop and implement state of the art methodologies to provide our clients with the most advanced and stable solution for their projects. To learn more about RSIP Vision’s activities in a wide range of industrial domains, please visit our project page.

Share

Share on linkedin
Share on twitter
Share on facebook

Related Content

Percutaneous Nephrolithotomy

PCNL – Planning and real-time navigation

Prostate Tumor Segmentation

Implementing AI to Improve PI-RADS Scoring

RAS Navigation

Tissue Sparing in Robotic Assisted Orthopedic Surgeries

Procedural Planning in urology

Procedural Planning in Urology

C Arm X-Ray Machine Scanner

Radiation Reduction in Robotic Assisted Surgeries (RAS) Using AI

Visible spectrum color

Hyperspectral Imaging for Robotic Assisted Surgery

Percutaneous Nephrolithotomy

PCNL – Planning and real-time navigation

Prostate Tumor Segmentation

Implementing AI to Improve PI-RADS Scoring

RAS Navigation

Tissue Sparing in Robotic Assisted Orthopedic Surgeries

Procedural Planning in urology

Procedural Planning in Urology

C Arm X-Ray Machine Scanner

Radiation Reduction in Robotic Assisted Surgeries (RAS) Using AI

Visible spectrum color

Hyperspectral Imaging for Robotic Assisted Surgery

Show all

RSIP Vision

Field-tested software solutions and custom R&D, to power your next medical products with innovative AI and image analysis capabilities.

Read more about us

Get in touch

Please fill the following form and our experts will be happy to reply to you soon

Recent News

PR – Intra-op Virtual Measurements in Laparoscopic and Robotic-Assisted Surgeries

PR – Non-Invasive Planning of Coronary Intervention

PR – Bladder Panorama Generator and Sparse Reconstruction Tool

PR – Registration Module for Orthopedic Surgery

All news
Upcoming Events
Stay informed for our next events
Subscribe to Our Magazines

Subscribe now and receive the Computer Vision News Magazine every month to your mailbox

 
Subscribe for free
Follow us
Linkedin Twitter Facebook Youtube

contact@rsipvision.com

Terms of Use

Privacy Policy

© All rights reserved to RSIP Vision 2021

Created by Shmulik

  • Our Work
    • title-1
      • Ophthalmology
      • Uncategorized
      • Ophthalmology
      • Pulmonology
      • Cardiology
      • Orthopedics
    • Title-2
      • Orthopedics
  • Success Stories
  • Insights
  • The company