Collaborative Exercise

From ENFSI Monopoly Programme 2011: Project S-FIVE
Jump to: navigation, search

S-FIVE Collaborative Exercise


Although the S-FIVE project ended on 31 Dec. 2015, you may still submit new CE results; please feel free to contact the Project Team! (last updated Aug. 2018)


Contents


Introduction

This webpage contains a description of a Collaborative Exercise aimed at making an anonymous comparison of image/video enhancement capabilities across Europe and the globe, trying to learn from this exercise and providing some first insights into future design of formal Proficiency Tests.
In other words, please note that this exercise has not been formally approved by ENFSI (DIWG/QCC) as an official collaborative exercise or proficiency test.

The questions included in this exercise are based on the results obtained from Survey1 that the S-FIVE project conducted earlier.

For returning your results, please use the Answer Form, and the file names specified below.
The original final deadline for submission of results was Fri., April 24.

Details on how to submit your results are available on the CE Submission Page.

Some questions have been labeled optional.
Please make a best effort to complete all (other) questions. If, for whatever reason, you are unable to complete some questions (lack of tools, time, ...), please return any results you may have obtained (and indicate why you have not answered some questions).

You can also download a PDF document that contains a snapshot of this webpage (Wed. 04 March 2015).

Please note that the provided video player archives contain executable/autorun files.
Please consider appropriate PC setup and security measures when running/installing this software.

Please note that by participating to this exercise you agree that the S-FIVE project team may (re)use and publish all information reported in the Answer Form, all submitted image enhancement results, or any other provided information. However/See also:

All obtained information will only be used in an anonymous manner as outlined in the Privacy Policy and Statement and the Disclaimer & Terms and Conditions of Service available on the main S-FIVE Project website.

If you experience any issues or would have any questions related to the exercise, please have a look at the Contact Us information.

 


Qn.1: Image extraction at best quality:

Within this question, the aim is to extract at best quality a specified frame from video footage recorded in different CCTV system file formats. Best quality should be understood to mean: minimum degradation with respect to its form within the source file, aiming for 1:1 pixel mapping. In other words, please submit the specified frames in this raw form without attempting to restore, e.g., true to life aspect ratio, or performing any other image operations.


Question 1A.

  • Download the Zip File: XVis TestE, 7m distance, high-elevation example and extract its contents to a new folder.
    A player and files required for it to run are provided in the folder 'Player'. The folder '20140508' contains a video file in the .drv format as well as .sig metadata file (at the bottom of the folder tree in the folder titled 'Normal'.)
  • Extract the first frame of the file (from timestamp 'PM 04:43:01 2014/04/28') at best quality (see definition given above) using either
  1. the replay software supplied and the procedure described below, and
    submit your result using the filename 'Q1A_ReplayDescribed_YOUR INITIALS', or,
  2. some other method (with or without the replay software) if a preferable result is achievable, and submit your result using the filename 'Q1A_OtherMethod_YOUR INITIALS'
  • Procedure: How to replay the footage using the Provided Replay Software:
    • The replay software can be run by double clicking 'DVR_Player.exe'. The software is self contained and does not require installation.
    • Click the 'Play' style button on the bottom of the GUI. A Window titled 'Data List' will appear
    • Click 'Open' at the top of this window. A new window titled 'Source' will appear. Select the appropriate drive and navigate to the folder titled 'Normal' which contains the .drv file.
    • Ensure that you double click on the folder icon against 'Normal' so the icon changes from a closed folder to an open folder (the contents of the folder will not be shown).
    • Click 'OK' to confirm the selection which closes the window titled 'Source'
    • Click 'Playback' at the bottom of the window titled 'Data List'. A new window titled 'Playback' will appear.
    • Click 'OK' at the top-right of the window titled 'Playback'. After a moment or two, the footage will begin playing back.


Question 1B

  • Unzip 'TestE_7mDistance_HighElevation.zip' to obtain the video file 'TestE_7mDistance_HighElevation.PAR'
  • Double click on 'DM-observer-1.18.59-installer.exe' to install the replay software on your computer. This will open an installation wizard. Select 'Next' for each step of the wizard, and 'Done' to complete the installation.
  • Extract the first frame from timestamp '4:43:30 PM GMT' at best quality (see definition given above) using
  1. the replay software supplied and the procedure described below, and
    submit your result using the filename 'Q1B_ReplayDescribed_YOUR INITIALS', or,
  2. some other method if a preferable result is achievable, and submit your result using the filename 'Q1B_OtherMethod_YOUR INITIALS'
  • Procedure: How to replay the footage using the Provided Replay Software:
    • Open the replay software via the 'Start' menu of your PC. Start/Programs/DM NetVu Observer\DM NetVu Observer
    • In the top left of the GUI, select 'File/Open Video'. A new window will appear titled 'Open'
    • Navigate to the video file of .PAR file format and click on 'Open'
    • In the window titled 'Import Video', Click on 'OK'. After a moment or two, the footage will begin playing back.


Question 1C

  • What is the frame size (in pixels) of each of the source file recordings given for Question 1A & Question 1B?
  • What are your conclusions about the raw, encoded quality of the provided video frame/sequence in Q1A and Q1B?


Qn.2: Super-Resolution:

In this question, the aim is to use the material available in the supplied video frames to create a single enhanced image using super-resolution enhancement such that the characters of the number plates are shown with greater clarity.

The number plates used in this exercise are composed of random numbers and letters from the UK number plate alphabet. The number and letter combinations will therefore not necessarily follow the legal format specified by any particular country.


Question 2A.

  • Submit your selected best result using the filename 'Q2A_SuperRes_YOURINITIALS'.


Question 2B.

  • Extract the contents of the zip file in a new folder. It contains a video file in the .dav format as well as a further zip file 'Player.zip' containing replay software.
  • Apply super-resolution methods to attempt to enhance the number plates as best as possible
  • Submit your selected best result using the filename 'Q2B_SuperRes_YOURINITIALS'.
    If you wish to submit multiple results, e.g., for various procedure or tools and/or for each license plate, use the filenames 'Q2B_SuperRes_ProcedureNUMBER_PlateNUMBER_YOURINITIALS' and adapt/extend the answer form so that each procedure/tool used and number plate can be identified uniquely and all your results can be reproduced by another person.
    The answer form will ask to identify number plates as follows: 1=closest number plate, ..., 5=most distant.
     
    How to number plates
    .
  • Using the Provided Replay Software:
    • Unzip the replay software to a new folder, and run by double clicking 'play.exe'. The software is self contained and does not require installation.
    • Click the 'Eject' style button on the left of the GUI to locate the .dav file and select 'open'


Question 2C (Optional).

  • Download the following Zip File: TestH, 7m distance, high-elevation example containing video footage and replay software and apply super-resolution to attempt to enhance the number plates as best as possible (See Question2B for guidelines on how to use the provided archive).
  • Submit your selected best result using the filename 'Q2C_SuperRes_YOURINITIALS'.
    If you wish to submit multiple results, e.g., for various procedure or tools and/or for each license plate, use the filenames 'Q2C_SuperRes_ProcedureNUMBER_PlateNUMBER_YOURINITIALS' and adapt/extend the answer form so that each procedure/tool used and number plate can be identified uniquely and all your results can be reproduced by another person.
    How to number the plates is illustrated in the previous question.


Qn.3: Focal Deblur Tests:

  • Download the following image containing letter charts positioned at different focal depths from the camera lens. As well as the title at the top, each chart contains 6 lines of random letters from the English Alphabet. Then:
  1. Provide your best enhancement results for the test chart titled '125cm' and
    submit your result using the filenames 'Q3_125cm_YOURINITIALS' or 'Q3_125cm_LineNUMBER_YOURINITIALS'
  2. Attempt to enhance the 4th line of letters in the test chart titled '75cm' and
    submit your result using the filenames 'Q3_75cm_Line4_YOURINITIALS'
  3. (Optional): Attempt to enhance the 5th line of letters in the test chart titled '75cm' and
    submit your result using the filenames 'Q3_75cm_Line5_YOURINITIALS'
  4. (Optional): Attempt to enhance any of the other test charts and lines and
    submit your result using the filenames 'Q3_CHARTDISTANCEcm_LineNUMBER_YOURINITIALS'


Focal Blur (Thumbnail View)]


Click here to download the image (Random_Focus_150cm_from_lens.JPG)


Qn.4: Motion Deblur Tests:

Question 4A.

  • Please try to enhance any text, logo's or number regions of interest in the image below (max. of 5 different text, logo, or number regions).
  • If you wish to provide results for different regions of interest in the image, please specify their coordinates and/or indicate the region in a small thumbnail, and use the filenames: “Q4A_RegionNUMBER_YOURINITIALS”
Question4A Image

Click here to download the image Qn4A.png


Question 4B.

  • Please try to enhance any text, logo's or number regions of interest in the image below (max. of 5 different text, logo, or number regions).
  • If you wish to provide results for different regions of interest in the image, please specify their coordinates and/or indicate the region in a small thumbnail, and use the filenames: “Q4B_RegionNUMBER_YOURINITIALS”
Question4B Image


Click here to download the image Qn4B.png



Copyright Notice

All image/video data provided through the S-FIVE project Collaborative Exercise webpage is considered to be copyright protected; (C) S-FIVE Project Team; http://s-five.eu/

The S-FIVE project team would like to thank the Metropolitan Police Service, London, UK, for all their efforts made in recording, analyzing and selecting this image material.


Please read the full Copyright Notice


Bonus CE

An additional exercise called the Bonus Collaborative Exercise and its discussion/solution is also available.


Please read the full Copyright Notice


History and important updates of or corrections to this page

08 June 2015: each of the 22 participants to the CE was notified of his/her anonymous ID (number) he/she has been assigned.

12 May 2015: The CE is now closed except for those participants that are about to send in their results. New requests for participation may still be honored by the Project Team (please send e-mail), but submitted results may not be included in the final presentation (of all other earlier results).

07 May 2015: Added final original image/video (C) notice.
Please read the full Copyright Notice

20 April 2015: Fixed server white listing of the CE Submission Page.

10 April 2015: Instructions on how to submit your CE results are now available on the CE Submission Page.

09 March 2015: TEMPORARY (C) notice: any and all image material available through the S-FIVE project website is considered to be (C) protected; until further notice you may not (re-)use the images/video sequences available on the S-FIVE webserver for any other purpose then to participate to this collaborative exercise and submit your results to the project team. The final (C) notice will be less restrictive.
04 March 2015: fix typo image caption Qn4B, add notice about participation=agreement on anonymous reuse of provided info, provide PDF-snapshot-document of this webpage.
03 March 2015: formal launch of this webpage.


Personal tools
Namespaces

Variants
    Actions