My Thoughts: Extract and Verify the text from image using Selenium WebDriver

Saturday, October 5

Extract and Verify the text from image using Selenium WebDriver

Firstly WebDriver does not support the functionality of extracting text from an image , at least as of now  :) .
So if we would like to extract and verify text from an image then we should use OCR (Optical Character Recognition) technology.

Coming to OCR , here is one of the nice article , and it says :

OCR software extracts all the information from the image into easily editable text format.Optical character recognition (OCR) is a system of converting scanned printed/handwritten image files into its machine readable text format. OCR software works by analyzing a document and comparing it with fonts stored in its database and/or by noting features typical to characters. 

There are good no.of free OCR software tools . If your preferred program is Java then you can use one of the Java OCR libraries to extract text from an image. I used ASPRISE OCR java library in this article. To work with  ASPRISE OCR library , follow the below simple two steps.

  1. Download "Asprise OCR" libraries , depending on the operating system you are using .
  2. Unzip the downloaded  folder and add the aspriseOCR jar file to your working directory . If you want you can download the single jar file from here .
  3. Also Copy the "AspriseOCR.dll" file from unzipped downloaded folder and save it  under "C:\Windows\System32" .
Now you are ready to go  :P .

Look at the below sample image .

And here is the sample code to read the text from above image :

import java.awt.Image;
import java.awt.image.BufferedImage;
import java.awt.image.RenderedImage;
import javax.imageio.ImageIO;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
import com.asprise.util.ocr.OCR;

public class ExtractImage {
 WebDriver driver;
  public void setUpDriver() {
   driver = new FirefoxDriver();
 public void start() throws IOException{
 /*Navigate to page
  * and get the image source attribute
 String imageUrl=driver.findElement(By.xpath("//*[@id='post-body-5614451749129773593']/div[1]/div[1]/div/a/img")).getAttribute("src");
 System.out.println("Image source path : \n"+ imageUrl);

 URL url = new URL(imageUrl);
 Image image =;
 String s = new OCR().recognizeCharacters((RenderedImage) image);
 System.out.println("Text From Image : \n"+ s);
 System.out.println("Length of total text : \n"+ s.length());
 /* Use below code If you want to read image location from your hard disk   
   BufferedImage image = File("Image location"));   
   String imageText = new OCR().recognizeCharacters((RenderedImage) image);  
   System.out.println("Text From Image : \n"+ imageText);  
   System.out.println("Length of total text : \n"+ imageText.length());   


Here is the output of the above program:

Image source path :

Never M2suse the O, ne
Who Likes You
Never Say Busy To Th,e One
Who Needs You
Never cheat The One
Who ReaZZy Trust You,
Never foJnget The One
Who Zways Remember You.

Length of total text :

Thats for now. Have a great weekend ..!! :)

Reference :


  1. Good post, but I find one minor fault with the example code. You never actually make use of WebDriver in there even though you instantiate it. Might as well just omit WebDriver from the example. Or make a better example that shows use of WebDriver to navigate to some page, fetch the URL of an image via it's image src attribute, then invoke the code from your example code on OCR above.

  2. Thanks for the input David. :)

    Even I felt the same . I will modify the code soon.

  3. Hi Vamshi ..for above code it gives me a Error to add "DevIL.dll" file however i also tried this after adding this dll but it wont work ..can u pls tell me that have u already added this dll or there is something else ....! :(

  4. Hi Niyati ,

    I didnot use "Devil.dll" file . The only extra jar file I have used for this program is "aspriseOCR.jar" .

    Please make sure you have added correct "aspriseOCR" jar file. For every OS and also 32/64 bit machines they have separate jar files. Download the correct file and try again. :)
    wish you luck :P

    Also check the below article:

  5. Got Your point. thank you so much for such a nice explanation :)

  6. Pavan SandhimediaFriday, October 18, 2013


    in "Internet Explorer" it not allow to click or write value in text box so can you give me some idea for that

  7. Pavan,

    Can you try "" version of IEDriver. I tried "" version of IEDriver and it is working fine for me . If you are still having an issue please send the url on which you are trying ?

  8. Hi Vamsi,

    Datepicker is displayed, but not selecting the date. It is entering the loop and I am able to print the date to the console.

    cell.findElement(By.linkText(String.valueOf(day))).click(); this particular code is not doing the required action.

  9. Hi Kannan,

    I am on Firefox 24 and using selenium 2.35 jars.Code is working for me without any issues . What version of firefox and jars you are using ?

    Did you modify the code in anyway like changing the website ?

  10. Hi Vamshi,

    I am using the browser and jars as you said above. I tried using this code for my application. So, I made it generic instead of clicking on a particular day. It was not working but changed the code as below and it started working.

    if (cell.getText().equals(String.valueOf(day))){ // day is the input day we are providing




    I would also like to know if you will be able to help me with reading PDF that is embedded within the same web page, instead getting downloaded to a separate browser or window? For example, when you sign in to a web application, you will get redirected to a page, where you will find the agreement PDF displayed inside a frame which can be downloaded or read and agreed to proceed.



  11. If PDF file file embedded within the webpage then try to get the exact location of PDF file from sourcecode .Hope below one will help you.

  12. I think the admin of this website is in fact working hard in support of his website, as here every data is
    quality based data.

    my blog post: sports

  13. Hi vamshi.. I 2 got the same error. Pls tell me how 2 resolve the above error..?

  14. Hi, I just followed the same instructions in this code but at the run time i get below error

    Exception in thread "main" java.lang.UnsatisfiedLinkError: no AspriseOCR in java.library.path
    at java.lang.ClassLoader.loadLibrary(Unknown Source)
    at java.lang.Runtime.loadLibrary0(Unknown Source)
    at java.lang.System.loadLibrary(Unknown Source)
    at com.asprise.util.ocr.OCR.loadLibrary(
    at com.asprise.util.ocr.OCR.(
    at Test2.main(

  15. Hi Srikanth , Its my bad . I forgot to add another step of adding "AspriseOCR.dll" file at "C:\Windows\System32" .

    Add a copy of AspriseOCR.dll file at C:\Windows\System32 and the errors will go off . :)

  16. Hi Anjala, Its my bad .
    I forgot to add another step of adding "AspriseOCR.dll" file at "C:\Windows\System32" .

    Add a copy of AspriseOCR.dll file at C:\Windows\System32 and the errors will go off . :)

  17. Thank you so much Vamshi, got the resolution. Also, we need to copy the following dll files[DevIL.dll, AspriseJTwain.dll, ILU.dll] as well into system32 location, otherwise we will get the below error:
    FAILED: start
    java.lang.UnsatisfiedLinkError: C:\Windows\System32\AspriseOCR.dll: Can't find dependent libraries
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary1(

  18. very can also do this with Onenote,Read this -

  19. Thanks a lot, it really helped me

  20. Hi vamshi,

    we can also use driver.findelement(By Locator).sendkeys(Keys.Return);

    to select the dates

  21. Hi Guys i tried all the ways specified above , but find the same error still , can any one help me out pls

    I have donwloaded the ASpire OCR zipped file and unzipped
    Copied the DLL files ( ASpireOCr.dll , DEVIL and iLU DLL to system32 folder)
    Added the aspireOCr jar file to eclipse external jar
    Now written a script to read the text inside The image

    I have added the Environment Path value as D:/Aspires - this folder consist of unzipped data
    also have added the Environment classpath D:\ASpires\aspriseOCR.jar

    Now when i run the script

    WebDriver fire = new FirefoxDriver();
    String urlOfImage=fire.findElement(By.xpath("//*[@id='post-body-5614451749129773593']/div[1]/div[1]/div/a/img")).getAttribute("src");

    System.out.println("image text"+urlOfImage);

    URL url = new URL(urlOfImage);
    Image i0 =;
    String s = new OCR().recognizeEverything((RenderedImage) i0); // exception got is here


    Exception in thread "main" java.lang.UnsatisfiedLinkError: no AspriseOCR in java.library.path

    Can any one please help me out in correcting it ?

  22. Can you please try adding "AspriseOCR.dll" file at "C:\Windows\System32" and run ?

  23. Hi,

    Nice Blog, but i have one question.
    Suppose i have one document then how can i select text from that document using selenium webdriver.
    Refer attached screenshot for example of select text from document.
    Please reply me ASAP.

  24. Can you add the screenshot once ?

  25. Is it possible to extract the text from a Scanned JPG file which is stored in the system using Selenium..?
    If there is any possibility , can u please post the code for that?

  26. Hi..thanx for ur great post..
    Is it possible to extract the text from a Scanned JPG file which is stored in the system using Selenium..?

    If there is any possibility , can u please post the code for that?

  27. HI,

    Last time i was attached screenshot you don't got screenshot?
    Refer attached screenshot.

  28. OCR technology has its own limitations. It always won't give good results that we are looking for (bad I know :) )

  29. Hey Akash,
    Just want to know why do you want to select this text ? :)
    Are you working any specific test case or you are exploring selenium in deep :)

    If you are looking for copy paste you can do it using Keys method.

  30. Please use this code wherever you found Two Jquery calendar (Here i used Hash table)

    Hashtable h=new Hashtable();
    h.put("January",0 );

    int expMonth;
    int expYear;

    // Calendar Month and Year
    String calMonth = null;
    String calYear = null;
    boolean dateNotFound;
    dateNotFound = true;
    expMonth= 5;
    expYear = 2014;


    calMonth = driver.findElement(By.className("ui-datepicker-month")).getText(); // get the text of month
    calYear = driver.findElement(By.className("ui-datepicker-year")).getText();

    if(((Integer)h.get(calMonth))+1 == expMonth && (expYear == Integer.parseInt(calYear)))
    String block="//div[@class='monthBlock first']/table/tbody/tr/td"; // THIS IS FIRST CALENDAR
    dateNotFound = false;
    // parseInt - Converts String to integer and indexof( It will return the index position of String)
    else if(((Integer)h.get(calMonth))+1 < expMonth && (expYear == Integer.parseInt(calYear)) || expYear > Integer.parseInt(calYear))
    String block="//div[@class='monthBlock last']/table/tbody/tr/td"; // THIS IS SECOND CALENDAR

    selectDate(expDate,block); // PASSING DATE AND CALENDAR
    dateNotFound = false; // Otherwise it will rotate continuously
    else if((Integer)h.get(calMonth)+1 > expMonth && (expYear == Integer.parseInt(calYear)) || expYear < Integer.parseInt(calYear))
    System.out.println(" Please enter the date greater than Current date");
    dateNotFound = false;



    public static void selectDate(String date,String block) throws IOException

    String monthblock=block;

    List dateWidget = driver.findElements(By.xpath(monthblock));

    for (WebElement cell: dateWidget)
    //Selects Date
    if (cell.getText().equals(date))




  31. Hi All,

    How can we select a date dynamically for the months which are not displayed in the view. Example, It's April and I want to select date from July which is not in the current datepicker view so how can we achieve that?