How to compare if the output of two different documents is the same?

add tag

samcarter

It is often interesting to know if the output of two different documents (or the same document using different package/kernel versions) is the same.

Which different techniques can be used to test this?

----

Just as an example document for testing:

```
\documentclass{beamer}

\begin{document}
	
\begin{frame}
\vspace{0.1mm}% test with and without this line
abc def
\end{frame}	
	
\end{document}
```

Top Answer

samcarter

One possible approach is to make a visual diff of the resulting documents.

The following script is rather hacky and has a lot of external dependencies, but maybe it can be used as a starting point.

The main idea is that each page of the documents gets converted into a pixelate image and then the pixel values are compared. If they differ, they are highlighted in red.

### Usage

It can be executed with 


```
lua pdfdiv.lua "document_old" "document_new"
```
and will produce an image `diff-<page number-1>.png` for every page that has differences

----

```
#!/usr/local/bin/lua

---------------------------------------------------------------------
--
-- making a visual diff of two pdfs
--
---------------------------------------------------------------------
function visual_div(old, new)

    -- stripping file ending from name
    doc_old = old:match("(.+).pdf") or old
    doc_new = new:match("(.+).pdf") or new

    -- getting number of pages in each document ---------------------
    local handle = io.popen("pdfinfo " .. doc_old .. ".pdf | grep Pages | awk '{print $2}'")
    local pages_old = handle:read("*a")
    handle:close()
    
    local handle = io.popen("pdfinfo " .. doc_new .. ".pdf | grep Pages | awk '{print $2}'")
    local pages_new = handle:read("*a")
    handle:close()

    ---- converting pdf's into png images ---------------------------
    os.execute("convert -density 300 " .. doc_old .. ".pdf " .. doc_old .. ".png")
    os.execute("convert -density 300 " .. doc_new .. ".pdf " .. doc_new .. ".png")

    -- loop over pages ----------------------------------------------
    for i = 0, pages_new-1, 1 
    do 

        -- check if page exists in old document ---------------------
        if( pages_old+0.0 >= i)
        then

            -- suffix based on number of pages ----------------------
            if( pages_old+0.0 == 1.0 )
            then 
                suffix = ""
            else
                suffix = "-" .. i
            end -- check if only one page

            -- calculating diff -------------------------------------
            os.execute("compare -compose src " .. doc_old .. suffix .. ".png " .. doc_new .. suffix .. ".png diff" .. suffix .. ".png")
            
            -- getting number of changed pixels ---------------------
            local handle = io.popen("convert diff" .. suffix .. ".png -print '%[fx:w*h*(1-mean)]'   null:")
            local diff_pixel = handle:read("*a")
            handle:close()
                
            -- clean up ---------------------------------------------
            os.execute("rm " .. doc_new .. suffix .. ".png")
            os.execute("rm " .. doc_old .. suffix .. ".png")
            
            -- keeping only images with differences -----------------
            if(diff_pixel+0.0 > 0.0)
            then
                -- print(diff_pixel)
            else
                os.execute("rm diff" .. suffix .. ".png")
            end

        end -- check if page exist

    end -- loop over pages

end -- end function

---------------------------------------------------------------------
--
-- calling function
--
---------------------------------------------------------------------

-- e.g. lua pdfdiv.lua "document_old" "document_new"
visual_div(arg[1], arg[2])
```

### Dependencies:

- `convert` and `compare` from the `ImageMagick` library
- pdfinfo
- grep
- awk
- whatever I forgot in the list above

### Disadvantages:

- slow (I really mean it, don't try with a very long document)

### Advantages:

- will not only show that something has changed, but provide visual context of what has changed. This way the user can for example judge if they mind that a word is 1 mm further down or not
- will sort out which pages actually do have changes, so no need to check all of them

### Output from the test document:

![diff.png](/image?hash=c44a8ed6385bf000487198b351819112d33d0a358225fcca1d8bf6144dd4e5e4)

Answer #2

samcarter

Ulrike Fischer made me aware of the `pdfpagediff` package

```
\documentclass{article}
\usepackage{pdfpagediff}

\begin{document}
\layerPages{document.pdf}{document2.pdf}
\end{document}
```

This will produce a new document with the two previous ones overlayed. If you have adobe reader, you can use the `layers` menu at the left hand site to toogle between the documents to see if there are any significant changes.

### Caveats: 

- If you have a document with opaque background (like beamer), you will only be able to see one layer at a time instead of both overlayed (but you will still be able to toggle between them)
- only works with pdfTeX

2 Answers