Python Made Easy With Posit Part 1: Reporting

Reporting with code is cumbersome

Automating reports using Python is often cumbersome, time-consuming, and challenging to maintain. This statement is especially true for complex, lengthy, or highly custom reports for a few reasons:

• Too much code: Nearly every report’s details must be defined, causing simple tasks to require a lot of code resulting in lengthy codebases and rigid processes.

• Different formats need different packages: PDF, Word, and PowerPoint are all generated using different packages. Rendering a single report to multiple formats requires disparate codebases.

• Finding Object Coordinates: Several Python packages require objects to be precisely placed within the document. Finding the table number of page coordinates is a hassle.

• Iterations are slow: Making changes and seeing the output requires the entire code base to be re-executed.

Time to create some simple reports to demonstrate what creating reports using Python entails. As you go through them, consider how they would scale with a full report containing content and custom formatting - hundreds of lines of code could be necessary, even if report templates are used.

from reportlab.lib.pagesizes import letter
from reportlab.pdfgen import canvas
from reportlab.lib.styles import getSampleStyleSheet, ParagraphStyle
from reportlab.lib.units import inch
from reportlab.platypus import SimpleDocTemplate, PageTemplate, Table, TableStyle, Spacer, Paragraph
from reportlab.platypus.frames import Frame
from reportlab.lib import colors
from functools import partial
from reportlab.lib.enums import TA_JUSTIFY

# Create Header and Footer
def header(canvas, doc, content):
    canvas.saveState()
    w, h = content.wrap(doc.width, doc.topMargin)
    content.drawOn(canvas, doc.leftMargin, doc.height + doc.bottomMargin + doc.topMargin - h)
    canvas.restoreState()

def footer(canvas, doc, content):
    canvas.saveState()
    w, h = content.wrap(doc.width, doc.bottomMargin)
    content.drawOn(canvas, doc.leftMargin, h)
    canvas.restoreState()

def header_and_footer(canvas, doc, header_content, footer_content):
    header(canvas, doc, header_content)
    footer(canvas, doc, footer_content)

# Create the PDF document
doc = SimpleDocTemplate("test.pdf", pagesize=letter,
                        rightMargin=72, leftMargin=72,
                        topMargin=72, bottomMargin=18)

# Create header and footer content
styles = getSampleStyleSheet()
header_text = Paragraph("TEST HEADER", styles['Normal'])
footer_text = Paragraph("TEST FOOTER", styles['Normal'])

# Create the page template with header and footer
frame = Frame(doc.leftMargin, doc.bottomMargin, doc.width, doc.height, id='normal')
head_foot_template = PageTemplate(id='test', frames=[frame], 
                                  onPage=partial(header_and_footer, 
                                                 header_content=header_text, 
                                                 footer_content=footer_text))
doc.addPageTemplates([head_foot_template])

# Create Page One Content
pg1 = []
styles.add(ParagraphStyle(name='Justify', alignment=TA_JUSTIFY))
pg1.append(Paragraph("This is not as easy as it should be!", styles["Normal"]))
pg1.append(Spacer(1, 10))
pg1.append(Paragraph("Seriously, this is too much code to do something very simple...and we'll need more to do more.", styles["Normal"]))
pg1.append(Spacer(1, 10))

# Create a Table on Page 1
tbl_data = [['00', '01', '02', '03', '04'],
            ['10', '11', '12', '13', '14'],
            ['20', '21', '22', '23', '24'],
            ['30', '31', '32', '33', '34']]
tbl = Table(tbl_data, 5 * [0.4 * inch], 4 * [0.4 * inch])
tbl.setStyle(TableStyle([('ALIGN', (1, 1), (-2, -2), 'CENTER'),
                         ('TEXTCOLOR', (1, 1), (-2, -2), colors.black),
                         ('VALIGN', (0, 0), (0, -1), 'TOP'),
                         ('TEXTCOLOR', (0, 0), (0, -1), colors.black),
                         ('ALIGN', (0, -1), (-1, -1), 'CENTER'),
                         ('VALIGN', (0, -1), (-1, -1), 'CENTER'),
                         ('TEXTCOLOR', (0, -1), (-1, -1), colors.black),
                         ('INNERGRID', (0, 0), (-1, -1), 0.25, colors.black),
                         ('BOX', (0, 0), (-1, -1), 0.25, colors.black)]))

pg1.append(tbl)

# Write the document to disk
doc.build(pg1)

from pptx import Presentation
from pptx.enum.shapes import MSO_SHAPE
from pptx.util import Inches
from pptx.enum.text import PP_ALIGN

# Create a presentation object
prs = Presentation()

# Create the first slide with a blank layout
slide1_layout = prs.slide_layouts[5]
slide1 = prs.slides.add_slide(slide1_layout)
slide1.shapes.title.text = "Test Slide"

# Add a textbox and insert text
textbox = slide1.shapes.add_textbox(Inches(2), Inches(1.5), Inches(6), Inches(1))
text_frame = textbox.text_frame
paragraph = text_frame.add_paragraph()
paragraph.text = "This is also too much work for a very simple report!"

# Add a table with 2 rows and 2 columns
x, y, cx, cy = Inches(2.5), Inches(2.5), Inches(2), Inches(1.5)
shape = slide1.shapes.add_table(2, 2, x, y, cx, cy)
table = shape.table

# Set column widths
table.columns[0].width = Inches(2.0)
table.columns[1].width = Inches(2.0)

# Write column headings
table.cell(0, 0).text = 'Col A'
table.cell(0, 1).text = 'Col B'

# Write body cells
table.cell(1, 0).text = 'AA'
table.cell(1, 1).text = 'BB'

# Center align the text in all table cells
for row in table.rows:
    for cell in row.cells:
        for paragraph in cell.text_frame.paragraphs:
            paragraph.alignment = PP_ALIGN.CENTER

# Save the presentation
prs.save('test.pptx')

from docx import Document
from docx.shared import Inches

# Create a new Word document
doc = Document()

# Create Header
section = doc.sections[0]
header = section.header
header_paragraph = header.paragraphs[0]
header_paragraph.text = 'TEST HEADER'

# Create Footer
footer = section.footer
footer_paragraph = footer.paragraphs[0]
footer_paragraph.text = 'TEST FOOTER'

# Create First Page Content
doc.add_heading('Wish I Used Quarto Instead!', level=1)
doc.add_paragraph('Why do we need so many steps to create a very simple Word document in Python?!')

# Add a Table with 3 rows and 3 columns
table = doc.add_table(rows=3, cols=3)
table.style = 'Table Grid'

# Populate the table
table.cell(0, 0).text = 'A'
table.cell(0, 1).text = 'B'
table.cell(0, 2).text = 'C'

table.cell(1, 0).text = 'AA'
table.cell(1, 1).text = 'BB'
table.cell(1, 2).text = 'CC'

table.cell(2, 0).text = 'AAA'
table.cell(2, 1).text = 'BBB'
table.cell(2, 2).text = 'CCC'

# Save the document
doc.save('test.docx')

Automating reports should be straightforward and not require a substantial amount of time. These examples have shown that this is not the case when using Python. That is not to say that these tools never have a place, but more often than not, they are not an ideal choice for report automation.

Quarto to the rescue

Ideally, the following would be achievable when automating reports:

• Add text, tables, images, figures, code, and charts in a consistent and simple manner
• Place objects within a grid system
• Execute code in individual cells within the IDE to preview code output
• Render to different document formats without creating disparate codebases
• Iterate through reports efficiently instead of re-executing the entire codebase
• Easily embed citations and cross-references
• Be able to reference and reuse content across reports
• Use with multiple IDEs

With Quarto, we can have all of these things!

Quarto is a publishing system that allows text and code to be seamlessly weaved together in a functional and easy-to-use package with many quality-of-life features. Quarto functions similarly to a Jupyter Notebook with added benefits such as saving to plain text, outputting to various formats, native layout options, and much more. Quarto can even render Jupyter Notebooks for those of you who are ride-or-die Jupyter!

Quarto is a powerful tool for automating reports because it eliminates the need for code to be used in creating and structuring the report. Instead, it uses code in its proper place to generate output like dynamic tables and charts. Quarto can render reports in PDF, HTML, PowerPoint, Word, or ePub format while effortlessly integrating with Python and other languages. Let’s take a quick look at how Quarto can be used to automate our reporting!

Quarto 101

Every Quarto document starts with a YAML header, where document-level options can be defined. For example, adding header and footer-linked images to each page would look like this:

format: pdf
mainfont: Times New Roman
fontsize: 12pt
margin-header: |
  ![Logo image](/img/logo.png)
margin-footer: |
  ![Logo image](/img/logo.png)

After setting up the report options, the report’s body is created using a combination of text, code, and whatever else is required. The example below shows text typed into the IDE and a code-generated chart. We can preview the chart before rendering the document by running the code cell - as we would in a Jupyter Notebook. Quarto special comments (#|) are used within the code cell to place the image and assign a caption.

Quarto comes built-in with special comments(#|), short-code ({ }), extensions, and pre-defined attributes that make report customization quick and easy. We won’t cover all of these incredible features in detail, but let’s take a quick look at the example below to see how we can use attributes to determine the size, layout, and spacing of several images. We can organize these images exactly how we like with one line of quasi-code.

::: {layout="[[1,1], [1]]"}
![Surus](surus.png)

![Hanno](hanno.png)

![Lin Wang](lin-wang.png)
:::

After finalizing our report, we may need to output it to multiple formats, which is done by changing the YAML header. For example, if we had a PDF with the following YAML header and wanted to change the format to word. We would go from this:

format: pdf
mainfont: Times New Roman
fontsize: 12pt

to this:

format: docx
mainfont: Times New Roman
fontsize: 12pt

After changing the YAML header, we can re-render the document and get our report in the newly defined format. A single Quarto file can automate a report output to one or multiple file formats. Quarto makes report automation both easy and efficient!

There is even more to Quarto

Quarto has many other awesome features we did not cover in detail, such as:

• Multi-language support (Python, R, Julia, Observable JS)
• Multiple IDE Support (VS Code, Jupyter Notebooks, RStudio)
• Parameterized reports
• Dynamic variables
• Support for interactive components (Shiny for Python, Plotly)
• Extensions
• Git compatibility
• and much more!

Be sure to explore the Quarto website to learn more and see fully developed report examples. Quarto makes reporting in Python easy. Don’t just take my word for it:

Sharing with Posit Connect

After automating a report, the next step is to execute and share it. Using Posit Connect, Quarto reports and other data products (applications, dashboards, datasets, APIs, etc.) can be shared and refreshed on a schedule, conditionally, or on demand. Data products can be published manually through IDE push-button publishing, the command line, Git-backed publishing, or API-backed publishing.

“Posit creates amazing software that makes it easy for me to do my job. Connect has utterly changed things at work. Whereas before I’d have to create something, then send people an output HTML, I now can hit publish, and the proper controls and access are there.”

Tori Oblad, Enterprise Data and Analytics Officer, WaFd Bank

Posit Connect can deploy:

• Quarto
• Jupyter Notebooks
• Shiny for Python
• Streamlit
• Dash
• Tensorflow Model APIs
• Vetiver
• Bokeh
• Flask
• Plumber
• FastAPI
• Pins
• Tableau Analytics Extensions