C H A P T E R  9

PL/SQL from SQL

by Adrian Billington

Functions are an integral part of any well-designed PL/SQL application. They are an embodiment of programming best practices, such as code modularization, reuse, and the encapsulation of business or application logic. When used as simple building-blocks for larger programs, they can be an elegant and simple way to extend functionality while reducing code-complexity at minimal cost.

Conversely, when PL/SQL functions are heavily used, particularly in SQL statements, there can be a range of associated costs, most notably for performance. Depending on the nature of the function, simply calling PL/SQL from SQL and/or excessive I/O can degrade performance of even the most trivial of queries.

Given this, I will describe a variety of costs associated with calling PL/SQL functions from SQL and, more importantly, demonstrate a range of techniques that you can use to reduce their impact. By the end of this chapter, you will have a greater awareness of the costs of using PL/SQL functions in your applications. You will therefore be better placed to make good design decisions for your new or re-factored applications and improve your performance tuning efforts on critical parts of your system.

images Note The examples in this chapter have all been run on an 11.2.0.2 Oracle Enterprise Linux database. The code listings are available for download in their entirety from the Apress web site. In many cases I have reduced the output to the minimum required to convey my message and maintain the flow of the chapter. All examples can be executed in SQL*Plus and the download pack includes any utilities that I’ve used in addition to those provided in the Oracle Database software.

The Cost of Using PL/SQL Functions in SQL

The SQL and PL/SQL languages are seamlessly integrated. Since Oracle Database 9i, the two languages have even shared the same parser. We use this inter-operability to our advantage in most of the code that we write, such as SQL cursors in PL/SQL programs or PL/SQL functions that are called from SQL. We often do this with little concern for what Oracle Database has to do under the covers to run our programs.

Despite this integration, there is a runtime cost to combining them, in particular when calling PL/SQL from SQL (the focus of this chapter, of course). I’m going to spend the next few pages exploring some of these costs. What is interesting is that Oracle has been working hard to reduce the impact of combining SQL and PL/SQL code over several versions, so the issues I describe are not necessarily in their final state.

My investigation of the costs of using PL/SQL functions in SQL can be broken down into three main areas:

  • Performance
  • Predictability (which is related to performance)
  • Side effects

images Note I do not cover syntactic or operational restrictions (such as DML being prohibited in PL/SQL functions called from SQL). These are not runtime costs as such and are well-documented elsewhere.

Context-Switching

Context-switching is an almost-irreducible cost of computing that occurs in CPUs, operating systems, and software. The mechanics differ depending on where it is occurring, but it is often a CPU-intensive operation that designers attempt to tune as much as is possible.

When we consider “context switching” in relation to Oracle Database, we specifically refer to the exchange of processing control between the SQL and PL/SQL engines (without necessarily understanding what occurs in such a process). These two engines are separate and distinct but we use them interchangeably. This means that when we call SQL from PL/SQL or vice versa, the calling context needs to store its process state and hand over control and data to its counterpart engine (which may or may not be picking up from an earlier switch). This switching cycle is computationally intensive and can typically be repeated so many times that its effects on response times can become quite noticeable.

For many, the first encounter with context-switching in Oracle Database would have been the introduction of the BULK COLLECT and FORALL PL/SQL features in Oracle 8i, designed specifically to reduce context-switching. These performance features made it possible to bind arrays of data between the SQL and PL/SQL engines in a single context-switch, dramatically reducing the number of context-switches that were generated by the row-by-row PL/SQL techniques that were prevalent at the time. For several major releases, almost all of the performance benefits of these language features could be attributed to the reduction in context-switching they facilitated. (In more recent releases, some further benefits such as redo optimization for bulk INSERT operations can also be realized when using PL/SQL array processing.)

images Note The reduction of context-switching is such an effective performance technique that the Oracle designers added an implicit BULK COLLECT for cursor-for-loops to the PL/SQL optimizing compiler introduced in Oracle Database 10g.

All of which is good news for PL/SQL programmers, of course, but the effects of excessive context-switching are still encountered elsewhere. In particular, context-switching is a performance penalty to pay for calling PL/SQL functions from SQL.

Measuring Context-Switching

As good as Oracle Database’s instrumentation is (and it is truly excellent), context-switching is not something that can be directly measured; it’s performed way down in the depths of the kernel and there is no simple statistic or counter available to quantify it. But given that context-switching is a fact of calling PL/SQL from SQL (and vice versa, though I shall cease to remind readers of this), its impact can be inferred with a simple set of measurements and supplied utilities.

Listing 9-1 shows a simple comparison between two SQL statements, one of which calls a PL/SQL function for every row. The function does nothing but return the input parameter and the timings are compared using the SQL*Plus timer. I’ve used an array size of 500 and Autotrace to suppress the output.

Listing 9-1. The Cost of Calling a PL/SQL Function from SQL (Baseline)

SQL> SELECT ROWNUM AS r
  2  FROM   dual
  3  CONNECT BY ROWNUM <= 1e6;

1000000 rows selected.

Elapsed: 00:00:01.74

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          0  consistent gets
          0  physical reads
          0  redo size
    6290159  bytes sent via SQL*Net to client
      40417  bytes received via SQL*Net from client
       2001  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
    1000000  rows processed

The baseline SQL statement generated 1 million rows in less than 2 seconds. As you can see from the Autotrace output, a SQL statement of this form is light on resources other than CPU (and network transfer and memory at larger rowcounts). Adding a PL/SQL function call to the SQL makes quite a difference to the elapsed time, as the Listing 9-2 demonstrates.

Listing 9-2. The Cost of Calling a PL/SQL Function from SQL (Using the Function)

SQL> CREATE FUNCTION plsql_function(
  2                  p_number IN NUMBER
  3                  ) RETURN NUMBER AS
  4  BEGIN
5     RETURN p_number;
  6  END plsql_function;
  7  /

Function created.

SQL> SELECT plsql_function(ROWNUM) AS r
  2  FROM   dual
  3  CONNECT BY ROWNUM <= 1e6;

1000000 rows selected.

Elapsed: 00:00:05.61

I’ve omitted the Autotrace output as it is almost identical to that of the baseline SQL, but you can see that adding a call to the function has added almost 4 seconds to the runtime of this simplest of queries.

As stated earlier, it is not possible to directly measure context-switching as such, but you can use tools such as SQL trace, the PL/SQL Hierarchical Profiler (an Oracle Database 11g feature), or the V$SESS_TIME_MODEL view to measure the time spent in the PL/SQL engine overall. Listing 9-3 shows the summary results of a Hierarchical Profiler session for the previous SQL.

Listing 9-3. The Cost of Calling a PL/SQL Function from SQL (Hierarchical Profiler Results)

FUNCTION                    LINE#      CALLS SUB_ELA_US FUNC_ELA_US
-------------------------- ------ ---------- ---------- -----------
__plsql_vm                      0    1000003    3122985     2006576
PLSQL_FUNCTION                  1    1000000    1116012     1116012
__anonymous_block               0          3        397         311
DBMS_OUTPUT.GET_LINES         180          2         86          82
DBMS_OUTPUT.GET_LINE          129          2          4           4
DBMS_HPROF.STOP_PROFILING      59          1          0           0

This profile clearly shows that the PL/SQL function was called 1 million times and this accounted for a little over 1 second of elapsed time. What is also interesting is the fact that the PL/SQL VM (the PL/SQL engine) spent 2 seconds of processing time outside of the function. Some of this time can be attributed to context-switching. Overall, the PL/SQL engine accounted for over 3 seconds of elapsed time. (Note that the DBMS_OUTPUT calls are a consequence of the serveroutput setting in SQL*Plus and are no more than background noise in this instance.)

images Note For a detailed discussion on the PL/SQL Hierarchical Profiler, see Chapter 13.

These figures are validated by an extended SQL trace of the query, as the TKProf output in Listing 9-4 demonstrates.

Listing 9-4. The Cost of Calling a PL/SQL Function from SQL (tkprof Output)

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch     2001      4.92       5.08          0          0          0     1000000
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total     2003      4.92       5.08          0          0          0     1000000

<snip>

Rows (1st)   Row Source Operation
----------   ---------------------------------------------------
   1000000   COUNT  (cr=0 pr=0 pw=0 time=3595520 us)
   1000000    CONNECT BY WITHOUT FILTERING (cr=0 pr=0 pw=0 time=1693263 us)
         1     FAST DUAL  (cr=0 pr=0 pw=0 time=28 us cost=2 size=0 card=1)

I’ve removed the Rows (max) and Rows (avg) statistics to shrink the output, but you can see from the rowsource statistics that it took approximately 3.6 seconds to project the 1 million rows for the resultset (this includes a small amount of network transfer time that was shown in the associated waits summary), yet it took just 1.7 seconds to generate the rows in the first place.

These examples are as simple as possible to demonstrate the cost of calling of PL/SQL functions from SQL. Despite their trivial timings, you can clearly see that the PL/SQL function adds a significant overhead to the response time (over 300%). Of course, PL/SQL functions that have much longer elapsed times (especially those functions that include SQL) will cause you far greater performance issues than context-switching problem. For PL/SQL-only functions (i.e. no SQL in the function body), however, context-switching can be a major part of the PL/SQL VM’s elapsed time.

Context-switching can also have a performance impact on queries that have plenty of other work to do, such as accessing and joining sets of data. In Listing 9-5, I have a lightweight PL/SQL function and I’ve used this to format part of a sales report. As before, I’ve used Autotrace and SQL*Plus timing to record the runtime statistics.

Listing 9-5. Context-Switching in More Intensive Queries

SQL> CREATE FUNCTION format_customer_name (
  2                  p_first_name IN VARCHAR2,
  3                  p_last_name  IN VARCHAR2
  4                  ) RETURN VARCHAR2 AS
  5  BEGIN
  6     RETURN p_first_name || ' ' || p_last_name;
  7  END format_customer_name;
  8  /

Function created.

SQL> SELECT t.calendar_year
  2  ,      format_customer_name(
  3            c.cust_first_name, c.cust_last_name
  4            )                 AS cust_name
  5  ,      SUM(s.quantity_sold) AS qty_sold
6  ,      SUM(s.amount_sold)   AS amt_sold
  7  FROM   sales     s
  8  ,      customers c
  9  ,      times     t
 10  WHERE  s.cust_id = c.cust_id
 11  AND    s.time_id = t.time_id
 12  GROUP  BY
 13         t.calendar_year
 14  ,      format_customer_name(
 15            c.cust_first_name, c.cust_last_name
 16            )
 17  ;

11604 rows selected.

Elapsed: 00:00:06.94

Statistics
----------------------------------------------------------
       3216  consistent gets
          0  physical reads

Note that I’ve trimmed the Autotrace output to include just the I/O statistics. The report completed in approximately 7 seconds. I’ve compared this to a SQL-only implementation in Listing 9-6.

Listing 9-6. Context-Switching in More Intensive Queries (SQL Implementation)

SQL> SELECT t.calendar_year
  2  ,      c.cust_first_name || ' ' || c.cust_last_name AS cust_name
  3  ,      SUM(s.quantity_sold)                         AS qty_sold
  4  ,      SUM(s.amount_sold)                           AS amt_sold
  5  FROM   sales     s
  6  ,      customers c
  7  ,      times     t
  8  WHERE  s.cust_id = c.cust_id
  9  AND    s.time_id = t.time_id
 10  GROUP  BY
 11         t.calendar_year
 12  ,      c.cust_first_name || ' ' || c.cust_last_name
 13  ;

11604 rows selected.

Elapsed: 00:00:01.40

Statistics
----------------------------------------------------------
       3189  consistent gets
          0  physical reads

By removing the function in favor of a SQL expression, the runtime of the report reduced to less than 1.5 seconds. You can see, therefore, that even when accessing and joining multiple rowsources, the presence of a PL/SQL function can still impact the overall response time of the query.

Other PL/SQL Function Types

The context-switching penalty applies to any PL/SQL source that is invoked from SQL (including user-defined aggregate functions, type methods, or table functions). A good example is the STRAGG function that Tom Kyte wrote many years back to demonstrate user-defined aggregate functions (then a new feature of Oracle 9i). Until Oracle Database 11g, there was no built-in equivalent to STRAGG, so this utility function became extremely popular as a solution to string-aggregation problems on Oracle forums.

The release of the LISTAGG built-in function provides a significant performance boost over STRAGG by simply removing the need for the SQL engine to perform context-switching. I’ve demonstrated this in Listing 9-7 with a simple example of aggregating the range of IDs for each customer in the supplied SH.CUSTOMERS table.

Listing 9-7. Built-in vs. User-Defined Aggregate Function

SQL> SELECT cust_first_name
  2  ,      cust_last_name
  3  ,      LISTAGG(cust_id) WITHIN GROUP (ORDER BY NULL)
  4  FROM   customers
  5  GROUP  BY
  6         cust_first_name
  7  ,      cust_last_name;

5601 rows selected.

Elapsed: 00:00:00.23

SQL> SELECT cust_first_name
  2  ,      cust_last_name
  3  ,      STRAGG(cust_id)
  4  FROM   customers
  5  GROUP  BY
  6         cust_first_name
  7  ,      cust_last_name;

5601 rows selected.

Elapsed: 00:00:01.40

I’ve suppressed the Autotrace output, but neither method incurs any physical I/O and the logical I/O is identical. Yet the user-defined aggregate function takes over six times longer than the built-in equivalent.

In summary, I’ve demonstrated that simply calling PL/SQL functions from SQL can be expensive, regardless of the nature of the function itself. Later, I’ll demonstrate various methods you can apply to reduce or remove this cost altogether from your SQL queries.

Executions

It is often said that the fastest way to do something is not to do it at all. It’s also commonly said that even when something is nanoseconds-fast, if you execute it enough times, its elapsed time will start to mount up. This is what commonly happens with PL/SQL functions that are executed from SQL. It follows that the more times a PL/SQL function is executed from a SQL statement, the greater the time and resources that are spent in both the function itself and the associated context-switching.

When a PL/SQL function is included in a SQL statement, how many times should that function be executed? There are a number of factors to be considered when attempting to answer this question; even assuming that you have some knowledge of the function’s usage in the query, you can’t be certain.

SQL is a non-procedural language, yet trying to predict the number of times that a PL/SQL function is executed by a SQL query is a decidedly procedural way of thinking. You have no way of controlling the innermost workings of the SQL engine and therefore no absolute control over how many times a PL/SQL function is called. (I say absolute because there are techniques you can use to reduce a function’s executions, but you can’t reliably predict the number of calls that you ultimately reduce it to.) If you don’t attempt to control the number of executions, then it follows that you won’t have any influence over the contribution that your function makes to the response time of your SQL statement.

Query Rewrite and the CBO

There are also the effects of the Cost Based Optimizer (CBO) to consider. With every release of Oracle Database, the CBO becomes more sophisticated (read complex); in particular, it finds more ways to optimize queries through transformation techniques. This means that the query block you call your PL/SQL function from might not exist in the final version of the query that the CBO generates.

This can easily be demonstrated with a small example of the CBO’s view-merging transformation. Listing 9-8 contains a mock report that aggregates product sales by customer and calendar year. The PL/SQL function in this example maintains a counter to trace the number of executions. It is called from an in-line view of the CUSTOMERS table in an attempt to pre-compute and also limit the number of function calls.

Listing 9-8. CBO Query Transformation and Function Executions

SQL> SELECT t.calendar_year
  2  ,      c.cust_name
  3  ,      SUM(s.quantity_sold) AS qty_sold
  4  ,      SUM(s.amount_sold)   AS amt_sold
  5  FROM   sales     s
  6  ,     (
  7         SELECT cust_id
  8         ,      format_customer_name (
  9                   cust_first_name, cust_last_name
 10                   ) AS cust_name
 11         FROM   customers
 12        )          c
 13  ,      times     t
 14  WHERE  s.cust_id = c.cust_id
 15  AND    s.time_id = t.time_id
16  GROUP  BY
 17         t.calendar_year
 18  ,      c.cust_name
 19  ;

11604 rows selected.

Having executed the report, I can retrieve the latest counter to see how many times the function was executed, as demonstrated in Listing 9-9. Note that you can refer to the source code on the Apress web site for implementation details of the COUNTER package and its use in the FORMAT_CUSTOMER_NAME function.

Listing 9-9. CBO Query Transformation and Function Executions (Displaying Function Calls)

SQL> exec counter.show('Function calls'),

Function calls: 918843

PL/SQL procedure successfully completed.

The PL/SQL function has been executed over 918,000 times, although my intention was that the in-line view would limit the function executions to just 55,500 (i.e. once per row in the CUSTOMERS table). This hasn’t worked, however, because the CBO has simply merged the inline view with the outer query block, and as a result, my PL/SQL function has been applied to a much larger rowsource than I intended. The execution plan in Listing 9-10 shows that the inline view has been factored out (i.e. merged) and the projection information clearly shows that the function calls have been applied at the very last step in the plan.

Listing 9-10. CBO Query Transformation and Function Executions (Projection of the PL/SQL Function)

----------------------------------------------------
| Id  | Operation                      | Name      |
----------------------------------------------------
|   0 | SELECT STATEMENT               |           |
|   1 |  HASH GROUP BY                 |           |
|   2 |   HASH JOIN                    |           |
|   3 |    PART JOIN FILTER CREATE     | :BF0000   |
|   4 |     TABLE ACCESS FULL          | TIMES     |
|   5 |    HASH JOIN                   |           |
|   6 |     TABLE ACCESS FULL          | CUSTOMERS |
|   7 |     PARTITION RANGE JOIN-FILTER|           |
|   8 |      TABLE ACCESS FULL         | SALES     |
----------------------------------------------------

Column Projection Information (identified by operation id):
-----------------------------------------------------------

   1 - "T"."CALENDAR_YEAR"[NUMBER,22],
       "FORMAT_CUSTOMER_NAME"("CUST_FIRST_NAME","CUST_LAST_NAME")[4000],
       SUM("S"."AMOUNT_SOLD")[22], SUM("S"."QUANTITY_SOLD")[22]
2 - (#keys=1) "T"."CALENDAR_YEAR"[NUMBER,22],
       "CUST_LAST_NAME"[VARCHAR2,40], "CUST_FIRST_NAME"[VARCHAR2,20],
       "S"."AMOUNT_SOLD"[NUMBER,22], "S"."QUANTITY_SOLD"[NUMBER,22]
   3 - "T"."TIME_ID"[DATE,7], "T"."TIME_ID"[DATE,7],
       "T"."CALENDAR_YEAR"[NUMBER,22]
   4 - "T"."TIME_ID"[DATE,7], "T"."CALENDAR_YEAR"[NUMBER,22]
   5 - (#keys=1) "CUST_LAST_NAME"[VARCHAR2,40],
       "CUST_FIRST_NAME"[VARCHAR2,20], "S"."AMOUNT_SOLD"[NUMBER,22],
       "S"."TIME_ID"[DATE,7], "S"."QUANTITY_SOLD"[NUMBER,22]
   6 - "CUST_ID"[NUMBER,22], "CUST_FIRST_NAME"[VARCHAR2,20],
       "CUST_LAST_NAME"[VARCHAR2,40]
   7 - "S"."CUST_ID"[NUMBER,22], "S"."TIME_ID"[DATE,7],
       "S"."QUANTITY_SOLD"[NUMBER,22], "S"."AMOUNT_SOLD"[NUMBER,22]
   8 - "S"."CUST_ID"[NUMBER,22], "S"."TIME_ID"[DATE,7],
       "S"."QUANTITY_SOLD"[NUMBER,22], "S"."AMOUNT_SOLD"[NUMBER,22]

View-merging, subquery unnesting, and other complex query transformations are not the only CBO-related issues that can affect the number of executions of your PL/SQL function. The CBO is also free to ignore the order that your tables or predicates are listed (although with the latter there is a notable exception, specific to PL/SQL functions, and I’ll be describing it later). The ordering of joins and predicates can have a dramatic effect on the cardinality of each step of an execution plan; this, in turn, will probably cause your PL/SQL function to be executed more, or less perhaps, than you expect.

Internal Optimizations

In addition to the numerous CBO optimizations, Oracle Database also has a range of internal optimizations that are applied at runtime, and trying to second-guess the behavior of these can often be an exercise in futility. Not only will they probably change with database version, they can also be dependent on a wide range of factors, some of which you can derive and others that are hidden from you.

Oracle Database’s optimization for DETERMINISTIC functions is one such example. Because a deterministic function will, by definition, always return the same result for a particular input value, Oracle Database can avoid re-executing the PL/SQL function if it encounters the same inputs. The efficiency of this optimization is, however, dependant on other factors, such as data type, data length, cardinality (i.e. the number of distinct values), the database version, and even environmental settings such as array fetch size. For a good discussion of how these factors can result in a wildly varying reduction in function executions, see Dom Brooks’ “Determining Deterministic” series of articles on http://orastory.wordpress.com.

Similar behavior is seen when PL/SQL function calls are encapsulated in scalar-subqueries (a commonly-used technique to reduce the impact of functions in SQL and one that I demonstrate later in this chapter). While the scalar subquery cache appears to be a more effective optimization than that for deterministic functions, it is still dependant on some of the same factors (such as data type, length, and order) in addition to the size of the internal cache (which varies by database version). See Cost-Based Oracle Fundamentals by Jonathan Lewis (Apress, 2005) for a more in-depth description of the internal hash table that supports the scalar subquery cache. The fact that this hash table is quite small means that collisions (i.e. cache “misses”) are almost inevitable on all but the more trivial set of input values. Because you can’t predict when these collisions will occur, you should accept that you can usually derive some benefit by using the scalar subquery cache, but not of a precisely quantifiable amount.

Suboptimal Data Access

What happens when a PL/SQL function contains a SQL statement? An extremely common design pattern that can be found in most large database applications is that of SQL lookups wrapped in PL/SQL functions. In general, the principles behind such “getter” functions are sound (i.e. the encapsulation of common lookups, reusable data access methods, etc.), but the performance implications of such design decisions can be devastating, especially when the PL/SQL functions are called from SQL statements.

When a SQL statement is wrapped in a PL/SQL function and called from SQL queries, the cost of context-switching is automatically doubled—it now occurs both inside and outside the function. Far more critical, however, is the fact that by wrapping SQL lookups in functions, the optimization choices that the CBO would otherwise have had if the lookup were in the main body of the calling SQL statement are effectively disabled. The impact that this can have on your queries’ elapsed times can be severe (it’s fair to say that the cost of context-switching pales into insignificance in these scenarios).

Consider the example in Listing 9-11. I have a PL/SQL function that returns an exchange rate between two currencies for a given date. Note that I’ve kept this deliberately simple for the purposes of the example.

Listing 9-11. Encapsulating SQL Lookups in PL/SQL Functions (Create PL/SQL Function)

SQL> CREATE FUNCTION get_rate(
  2                  p_rate_date IN rates.rate_date%TYPE,
  3                  p_from_ccy  IN rates.base_ccy%TYPE,
  4                  p_to_ccy    IN rates.target_ccy%TYPE
  5                  ) RETURN rates.exchange_rate%TYPE AS
  6     v_rate rates.exchange_rate%TYPE;
  7  BEGIN
  8     SELECT exchange_rate INTO v_rate
  9     FROM   rates
 10     WHERE  base_ccy   = p_from_ccy
 11     AND    target_ccy = p_to_ccy
 12     AND    rate_date  = p_rate_date;
 13     RETURN v_rate;
 14  END get_rate;
 15  /

Function created.

In the report of sales by product and calendar year shown in Listing 9-12, I’ve used this function to calculate revenue in GBP as well as USD. Note that I’ve used Autotrace to suppress the resultset and enabled the SQL*Plus timer.

Listing 9-12. Encapsulating SQL Lookups in PL/SQL Functions (SQL with PL/SQL)

SQL> SELECT t.calendar_year
  2  ,      p.prod_name
  3  ,      SUM(s.amount_sold)                                     AS amt_sold_usd
  4  ,      SUM(s.amount_sold * get_rate(s.time_id, 'USD', 'GBP')) AS amt_sold_gbp
  5  FROM   sales     s
  6  ,      products  p
  7  ,      times     t
  8  WHERE  s.prod_id = p.prod_id
9  AND    s.time_id = t.time_id
 10  GROUP  BY
 11         t.calendar_year
 12  ,      p.prod_name
 13  ;

272 rows selected.

Elapsed: 00:01:05.71

The elapsed time of this query is roughly 66 seconds—a poor response time for an aggregate query over less than 1 million rows. The Autotrace output for this statement is provided in Listing 9-13.

Listing 9-13. Encapsulating SQL Lookups in PL/SQL Functions (Autotrace Output for SQL with PL/SQL)

Statistics
----------------------------------------------------------
     918848  recursive calls
         12  db block gets
    1839430  consistent gets
          0  physical reads
       3196  redo size
      16584  bytes sent via SQL*Net to client
        449  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        272  rows processed

As highlighted, that’s over 1.8 million consistent gets for the report (and over 918,000 recursive calls relating to the execution of the SQL statement embedded in the GET_RATE function). In Listing 9-14, I’ve converted this query into a “straight SQL” alternative by joining directly to the RATES table and calculating the currency conversion in an expression.

Listing 9-14. Encapsulating SQL Lookups in PL/SQL Functions (SQL-only Solution)

SQL> SELECT t.calendar_year
  2  ,      p.prod_name
  3  ,      SUM(s.amount_sold)                 AS amt_sold_usd
  4  ,      SUM(s.amount_sold*r.exchange_rate) AS amt_sold_gbp
  5  FROM   sales     s
  6  ,      times     t
  7  ,      products  p
  8  ,      rates     r
  9  WHERE  s.time_id        = t.time_id
 10  AND    s.prod_id        = p.prod_id
 11  AND    r.rate_date  (+) = s.time_id
 12  AND    r.base_ccy   (+) = 'USD'
 13  AND    r.target_ccy (+) = 'GBP'
14  GROUP  BY
 15         t.calendar_year
 16  ,      p.prod_name
 17  ;

272 rows selected.

Elapsed: 00:00:00.97

The performance difference is staggering! By removing the PL/SQL function and joining directly to the RATES table, the response time has dropped from 66 seconds to just 1 second.

images Note An unusual “feature” of PL/SQL functions called from SQL is that unhandled NO_DATA_FOUND exceptions silently return NULL rather than propagate the error. I haven’t explicitly handled the NO_DATA_FOUND exception in the GET_RATE function in Listing 9-11, so my query in Listing 9-12 won’t fail, even if the exchange rate I’m looking up doesn’t exist. Rather, the function will return NULL and my query will be lossless (in terms of rows). To emulate this in the SQL-only version of the query in Listing 9-14, I’ve simply used an outer join to the RATES table.

The Autotrace statistics for the SQL-only report are shown in Listing 9-15.

Listing 9-15. Encapsulating SQL Lookups in PL/SQL Functions (Autotrace Output for SQL-only Report)

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
       1758  consistent gets
          0  physical reads
          0  redo size
      16584  bytes sent via SQL*Net to client
        425  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        272  rows processed

The Autotrace output highlights a striking difference in the logical I/O required to satisfy the two SQL statements. The PL/SQL function incurs a penalty of over 1.8 million consistent gets. The TKProf report in Listing 9-16 clearly shows the cost of encapsulating the rates lookup in a function.

Listing 9-16. Encapsulating SQL Lookups in PL/SQL Functions (tkprof Output)

SELECT t.calendar_year
,      p.prod_name
,      SUM(s.amount_sold)                                     AS amt_sold_usd
,      SUM(s.amount_sold * get_rate(s.time_id, 'USD', 'GBP')) AS amt_sold_gbp
FROM   sales     s
,      products  p
,      times     t
WHERE  s.prod_id = p.prod_id
AND    s.time_id = t.time_id
GROUP  BY
       t.calendar_year
,      p.prod_name

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch        2    105.53     106.38          0       1734          0         272
------- ------  -------- ---------- ---------- ---------- ----------  ----------
<snipped output>

********************************************************************************
SQL ID: ajp04ks60kqx9 Plan Hash: 139684570

SELECT EXCHANGE_RATE
FROM
 RATES WHERE BASE_CCY = :B3 AND TARGET_CCY = :B2 AND RATE_DATE = :B1


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        0      0.00       0.00          0          0          0           0
Execute 918843     12.50      13.85          0          0          0           0
Fetch   918843      9.74      10.86          0    1837686          0      918843
------- ------  -------- ---------- ---------- ---------- ----------  ----------

The TKProf report shows that the SQL inside the PL/SQL function is treated as a separate query; in this case it has accounted for all of the additional logical I/O that the sales query generated and almost half of the additional runtime (if you’re wondering where the remaining runtime was spent for this example, a Hierarchical Profiler report showed that almost 59 seconds were spent in the PL/SQL engine overall. Some 47 seconds of this PL/SQL time was accounted for by the rates-lookup SQL itself—double that reported by the SQL trace above—meaning that approximately 23 seconds was PL/SQL overhead). The base query itself has incurred very little logical I/O and is in fact comparable to the SQL-only implementation.

This separation highlights an important point. If the embedded SQL is treated as a query in its own right, the CBO cannot optimize it in the context of the outer query. The CBO is completely ignorant of what the function actually does, so the RATES table does not even feature in the CBO’s decision-making. The results, as you can see, can be catastrophic.

You can see, therefore, that it makes good sense (from a performance perspective) to convert PL/SQL lookup functions to SQL. If performance is critical and such lookup functions are heavily used in your application, you are encouraged to remove them wherever possible. There are alternative encapsulation methods that I’ll demonstrate later in this chapter that can help you with this.

However, I recognize that it’s not always possible to “sweep” a legacy application to convert its SQL (it might be embedded in many different application tiers and technologies, or the regression testing overhead might be too prohibitive). In such cases, there are methods you can use to reduce the costs of the PL/SQL functions and again I’ll show some of these later.

Optimizer Difficulties

Many of the examples so far have shown PL/SQL functions called from the SELECT block (projection) of SQL statements but it is equally common to see user-written functions being used in predicates. This can give rise to a range of issues. Firstly, as I demonstrated earlier, a PL/SQL function used in a predicate will be executed an unpredictable number of times. Secondly, a PL/SQL function (or indeed any function) applied to an indexed column will prevent the use of the index. Thirdly, the CBO knows very little about PL/SQL functions as it usually has no statistics with which to work. This can lead to suboptimal execution plans. I will concentrate on the third issue here, as I described the first issue earlier and the second is well understood by the Oracle DBA/developer community.

PL/SQL Functions and Statistics

Why does the CBO have difficulties with PL/SQL functions? The answer is statistics. You know that the CBO relies on good-quality statistics to enable it to make sensible optimization decisions. When it comes to PL/SQL functions, however, the CBO almost always has no statistics to work with and has to resort to default heuristics instead. For example, when encountering a predicate of the form WHERE plsql_function(expression-involving-some-column) = value, the CBO defaults the predicate’s selectivity to just 1% (in the absence of statistics for the function, it has no alternative).

In Listing 9-17, I demonstrate this heuristic using a small table of 1,000 rows, a PL/SQL function, and a simple query with CBO tracing enabled (10053 event).

Listing 9-17. Default Selectivity for PL/SQL Function Predicate

SQL> CREATE TABLE thousand_rows
  2  AS
  3     SELECT ROWNUM           AS n1
  4     ,      RPAD('x',50,'x') AS v1
  5     FROM   dual
  6     CONNECT BY ROWNUM <= 1000;

Table created.

SQL> BEGIN
  2     DBMS_STATS.GATHER_TABLE_STATS(user, 'THOUSAND_ROWS'),
  3  END;
  4  /

PL/SQL procedure successfully completed.

SQL> CREATE FUNCTION plsql_function (
  2                  p_id IN INTEGER
  3                  ) RETURN INTEGER AS
  4  BEGIN
  5     RETURN 0;
  6  END plsql_function;
  7  /

Function created.
SQL> ALTER SESSION SET EVENTS '10053 trace name context forever, level 1';

Session altered.

SQL> set autotrace traceonly explain

SQL> SELECT *
  2  FROM   thousand_rows
  3  WHERE  plsql_function(n1) = 0;

Execution Plan
----------------------------------------------------------
Plan hash value: 4012467810

-----------------------------------------------------------------------------------
| Id  | Operation         | Name          | Rows  | Bytes | Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |               |    10 |   550 |     5   (0)| 00:00:01 |
|*  1 |  TABLE ACCESS FULL| THOUSAND_ROWS |    10 |   550 |     5   (0)| 00:00:01 |
-----------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("PLSQL_FUNCTION"("N1")=0)

SQL> ALTER SESSION SET EVENTS '10053 trace name context off';

Session altered.

In the absence of statistics for the PL/SQL function, the CBO has defaulted to a 1% selectivity to determine the cardinality of the predicate. You can see this reflected in the cardinalities (Rows) displayed in the execution plan above. An excerpt from the 10053 trace file in Listing 9-18 also demonstrates the CBO’s calculations.

Listing 9-18. Default Selectivity for PL/SQL Function Predicate (10053 Trace File)

***************************************
BASE STATISTICAL INFORMATION
***********************
Table Stats::
  Table: THOUSAND_ROWS  Alias: THOUSAND_ROWS
    #Rows: 1000  #Blks:  12  AvgRowLen:  55.00  ChainCnt:  0.00
Access path analysis for THOUSAND_ROWS
***************************************
SINGLE TABLE ACCESS PATH
  Single Table Cardinality Estimation for THOUSAND_ROWS[THOUSAND_ROWS]
  No statistics type defined for function PLSQL_FUNCTION
  No default cost defined for function PLSQL_FUNCTION
  No statistics type defined for function PLSQL_FUNCTION
  No default selectivity defined for function PLSQL_FUNCTION
  Table: THOUSAND_ROWS  Alias: THOUSAND_ROWS
Card: Original: 1000.000000  Rounded: 10  Computed: 10.00  Non Adjusted: 10.00
  Access Path: TableScan
    Cost:  5.10  Resp: 5.10  Degree: 0
      Cost_io: 5.00  Cost_cpu: 3285657
      Resp_io: 5.00  Resp_cpu: 3285657
  Best:: AccessPath: TableScan

I’ve highlighted some important information in this trace file. It states that there are no default statistics or statistics types defined for either cost or selectivity of the PL/SQL function. This shows you, therefore, that it is possible to provide statistics on PL/SQL functions; I’ll demonstrate how to do this later.

Predicate Ordering

When statistics for PL/SQL functions are missing, other potential issues can arise as a result of the order in which the predicates are coded. For example, if you write a WHERE clause that contains two PL/SQL function predicates, in the absence of statistics the functions will be executed simply in the order in which they appear in the SQL.

I’ve demonstrated this in Listing 9-19, where I have a SQL statement that filters on two PL/SQL functions, SLOW_FUNCTION and QUICK_FUNCTION. As their names suggest, one function is more costly than the other, so I’ve executed the SQL statement twice with the predicates in different orders, using Autotrace and a SQL*Plus timer to compare the results.

Listing 9-19. PL/SQL Functions and Predicate Ordering

SQL> CREATE FUNCTION quick_function (
  2                  p_id IN INTEGER
  3                  ) RETURN INTEGER AS
  4  BEGIN
  5     RETURN MOD(p_id, 1000);
  6  END quick_function;
  7  /

Function created.

SQL> CREATE FUNCTION slow_function (
  2                  p_id IN INTEGER
  3                  ) RETURN INTEGER AS
  4  BEGIN
  5     DBMS_LOCK.SLEEP(0.005);
  6     RETURN MOD(p_id, 2);
  7  END slow_function;
  8  /

Function created.

SQL> set autotrace traceonly

SQL> SELECT *
2  FROM   thousand_rows
  3  WHERE  quick_function(n1) = 0
  4  AND    slow_function(n1) = 0;

1 row selected.

Elapsed: 00:00:00.02

Execution Plan
----------------------------------------------------------
Plan hash value: 4012467810

-----------------------------------------------------------------------------------
| Id  | Operation         | Name          | Rows  | Bytes | Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |               |     1 |    55 |     5   (0)| 00:00:01 |
|*  1 |  TABLE ACCESS FULL| THOUSAND_ROWS |     1 |    55 |     5   (0)| 00:00:01 |
-----------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("QUICK_FUNCTION"("N1")=0 AND "SLOW_FUNCTION"("N1")=0)


SQL> SELECT *
  2  FROM   thousand_rows
  3  WHERE  slow_function(n1) = 0
  4  AND    quick_function(n1) = 0;

1 row selected.

Elapsed: 00:00:11.09

Execution Plan
----------------------------------------------------------
Plan hash value: 4012467810

-----------------------------------------------------------------------------------
| Id  | Operation         | Name          | Rows  | Bytes | Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |               |     1 |    55 |     5   (0)| 00:00:01 |
|*  1 |  TABLE ACCESS FULL| THOUSAND_ROWS |     1 |    55 |     5   (0)| 00:00:01 |
-----------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("SLOW_FUNCTION"("N1")=0 AND "QUICK_FUNCTION"("N1")=0)

I’ve removed the Autotrace statistics but kept the Explain Plan sections to show the order in which the PL/SQL functions were executed for each SQL statement. You can see that this matches the order in which they were coded and that this made quite a difference to the relative performance of the two statements. This is the point. The CBO has no idea that SLOW_FUNCTION has a lower selectivity or that it takes longer to execute than QUICK_FUNCTION. In the absence of supplied statistics, how can it? It makes no sense for the CBO to re-order the predicates, because it has no basis for doing so.

The good news is that there are several methods for generating or setting statistics for PL/SQL functions to mitigate this issue. These are described in the section entitled “Assisting the CBO.”

The Read-Consistency Trap

Listing 9-13 clearly demonstrates the resource-cost of putting SQL lookups into PL/SQL functions (and these costs are easy to quantify). However, there is another important pitfall of using such a design, although it is much harder to quantify the potential impact that this might have on your application(s). That is, the SQL inside the PL/SQL function is not read-consistent with the main SQL statement that is calling the function.

The TKProf report from Listing 9-12 (shown in Listing 9-16) demonstrated that the SQL inside the PL/SQL function was treated as a separate statement. This means that under the READ COMMITTED isolation level, it has its own read-consistency and not that of the SQL statement that calls the function.

In Listings 9-20 through 9-23, I’ve demonstrated this read-consistency trap. I have two sessions; the first of which (Session One) executes a long-running query that includes a call to a PL/SQL lookup function. The second session (Session Two) updates one of the lookup records at a random point in time, but importantly, this is while the first session’s query is still running.

I’ll begin with the PL/SQL lookup function in Listing 9-20.

Listing 9-20. The Read-Consistency Trap (Create PL/SQL Function)

SQL> CREATE FUNCTION get_customer_name (
  2                  p_cust_id IN customers.cust_id%TYPE
  3                  ) RETURN VARCHAR2 AS
  4     v_customer_name VARCHAR2(200);
  5  BEGIN
  6     SELECT cust_first_name || ' ' || cust_last_name
  7     INTO   v_customer_name
  8     FROM   customers
  9     WHERE  cust_id = p_cust_id;
 10     RETURN v_customer_name;
 11  END get_customer_name;
 12  /

Function created.

You can see that the lookup function fetches a customer name by primary key. In addition, I have created a small SLEEPER function to enable me to slow down Session One’s query and simulate a long-running SQL statement (this function simply calls DBMS_LOCK.SLEEP and returns a constant 0).

Continuing with the example, Session One issues the following query shown in Listing 9-21.

Listing 9-21. The Read-Consistency Trap (Session One’s Long-Running Query)

Session One
-----------
22:47:23 SQL> SELECT s.cust_id
22:47:23   2  ,      get_customer_name(s.cust_id) AS cust_name
22:47:23   3  ,      sleeper(1)                   AS sleep_time
22:47:23   4  FROM   sales s
22:47:23   5  WHERE  ROWNUM <= 10;

Note the time at which Session One issues this SQL. The 10-row resultset and the 1 second sleep for each row means that this query should complete in just over 10 seconds. While this is executing, Session Two updates one of the customer names that Session One is repeatedly querying, as shown in Listing 9-22.

Listing 9-22. The Read-Consistency Trap (Session Two’s Customer Update)

Session Two
-----------
22:47:26 SQL> UPDATE customers
22:47:26   2  SET    cust_last_name = 'Smith'
22:47:26   3  WHERE  cust_id = 2;

1 row updated.

22:47:26 SQL> COMMIT;

Commit complete.

Again, note the time that this update is committed (approximately 3 seconds into Session One’s long-running query). Listing 9-23 shows the effect of this on Session One’s report.

Listing 9-23. The Read-Consistency Trap (Session One’s Resultset)

Session One
-----------
   CUST_ID CUST_NAME                      SLEEP_TIME
---------- ------------------------------ ----------
         2 Anne Koch                               1
         2 Anne Koch                               1
         2 Anne Koch                               1
         2 Anne Koch                               1
         2 Anne Smith                              1
         2 Anne Smith                              1
         2 Anne Smith                              1
         2 Anne Smith                              1
         2 Anne Smith                              1
         2 Anne Smith                              1

10 rows selected.

Elapsed: 00:00:10.02

Approximately halfway through Session One’s resultset, the customer name for ID=2 changed, because the lookup function operates outside of the main query’s read-consistent image of data. The report itself is therefore inconsistent.

What does this mean if you use such PL/SQL functions? Clearly you are at risk of inconsistent resultsets every time you include the function in a query, particularly if your users can update the lookup table at any time. You would need to assess this risk in the context of your own application patterns, of course. Users who received an inconsistent report would naturally question what they were seeing, and even worse, question your application.

Unfortunately, the options for protecting yourself against this are less than satisfactory. Short of removing all PL/SQL lookup functions, you have a range of options, such as

  • Using the SERIALIZABLE isolation level.
  • Using SET TRANSACTION READ ONLY.
  • Making your PL/SQL function use flashback query with a given SCN to match the start of the main query.
  • Making your readers block writers with FOR UPDATE queries (which even then would only work if the lookup table was accessed directly elsewhere in your main query. At which point you’d need to question the need for your PL/SQL lookup function in the first place!)

Any of these options would solve the issue but would more likely cause you greater headaches by seriously limiting your application’s concurrency and performance.

Other Issues

I’ve described in some detail the major issues you need to be aware of when using PL/SQL functions in SQL. Needless to say, there are others and perhaps you’ve encountered a particular corner-case that I haven’t thought of. I’ll complete this section with a very brief mention of other issues that you might encounter at some stage.

Parallel Query

PL/SQL functions can be enabled to work with Oracle Database’s parallel query option (PQ). While many PL/SQL functions can be executed by PQ slaves without issue, it is sometimes possible to disable PQ by referencing a PL/SQL function that is not explicitly parallel-enabled. If you are having trouble parallelizing a SQL statement and it contains a PL/SQL function, check to see if the function is parallel-enabled. Assuming your PQ hints, session PQ settings, and PQ system parameters are valid for parallel query yet it’s simply not “taking,” re-create your PL/SQL function to include the PARALLEL_ENABLE clause and try again. Note that for a PL/SQL function to be parallel-enabled, it should not reference session state variables that a PQ-slave might not be able to access (such as package state).

NO_DATA_FOUND

I described this earlier in the commentary for Listing 9-12 but it’s worth reiterating for completeness. A NO_DATA_FOUND exception raised in a PL/SQL function called from a SQL statement will not propagate the exception! Instead, the function will return NULL.

Reducing the Cost of PL/SQL Functions

I’ve just spent several pages describing many of the difficulties, costs, and pitfalls that you might encounter when using PL/SQL functions. Yet this is a book for PL/SQL professionals, so I’m going to spend the remainder of this chapter describing a range of techniques to reduce the impact of using PL/SQL functions.

A Sense of Perspective

Before I continue, I think it is important to give a sense of perspective. Nothing in the world of Oracle development and performance is absolute and, fortunately, there are times when a basic implementation of a PL/SQL function will be more than adequate. Listings 9-24 and 9-25 demonstrate such an example. I’m going to create an isNumber checker to validate in-bound raw data and compare a PL/SQL function implementation with a SQL-only version. To do this, I’ve created an external table of seven fields (all as VARCHAR2(30)), dumped the data in the SALES table to a flat-file and modified 1 in every 10,000 AMOUNT_SOLD values to include non-numeric characters (AMOUNT_SOLD maps to FIELD_07 in the external table). Finally, I’ve created the IS_NUMBER function shown in Listing 9-24 to validate that strings are in fact of numeric format.

Listing 9-24. Validating Numeric Data (IS_NUMBER Function)

SQL> CREATE FUNCTION is_number (
  2                  p_str IN VARCHAR2
  3                  ) RETURN NUMBER IS
  4     n NUMBER;
  5  BEGIN
  6     n := TO_NUMBER(p_str);
  7     RETURN 1;
  8  EXCEPTION
  9     WHEN VALUE_ERROR THEN
 10        RETURN 0;
 11  END;
 12  /

Function created.

In Listing 9-25, I’ve used two alternative approaches to validate that the data in FIELD_07 is of numeric format. First, I’ve used the IS_NUMBER PL/SQL function (which returns 1 if the input value is a number). Second, I’ve used a SQL built-in regular expression, taken from an example in an OTN forum thread. I’ve used Autotrace to suppress the output and the SQL*Plus timer to record the timings.

Listing 9-25. Validating Numeric Data (PL/SQL Function vs. SQL Built-in)

SQL> SELECT *
  2  FROM   external_table
  3  WHERE  is_number(field_07) = 1;

918752 rows selected.
Elapsed: 00:00:10.19

SQL> SELECT *
  2  FROM   external_table
  3  WHERE  REGEXP_LIKE(
  4            field_07,
  5            '^( *)(+|-)?((d*[.]?d+)|(d+[.]?d*)){1}(e(+|-)?d+)?(f|d)?$',
  6            'i'),

918752 rows selected.

Elapsed: 00:00:28.25

You can see that the PL/SQL function implementation is much quicker (almost three times) than the regular expression. It is better still when reversing the two methods to find the small number of non-numeric records. Regular expressions are very CPU-intensive; compare the CPU times for the two SQL statements in V$SQL or trace the two queries with DBMS_UTILITY.GET_CPU_TIME to see the difference for yourself.

This example shows that the validity of using PL/SQL functions in SQL statements cannot be dismissed outright. Your requirements, your knowledge of the systems you are developing, and your testing will help you to determine whether a PL/SQL function is viable or not. And if it is viable, the techniques I will describe in the remainder of this chapter should help you to get a lot more from your implementations.

Using SQL Alternatives

It is, however, quite rare that a PL/SQL function will be quicker than a SQL-only expression or built-in function. You should consider whether it is possible or desirable to change some of your application’s critical queries (that use PL/SQL functions) to use SQL-only alternatives. If you are developing new applications, you have a good opportunity to build-in performance from the outset.

Some considerations will be required here, of course, and if any of the following are true, you might decide to favour some PL/SQL functions over a SQL implementation:

  • A PL/SQL function is not heavily used.
  • A PL/SQL function is too complicated to be easily expressed in SQL.
  • A SQL statement is not displaying any of the issues I’ve described.
  • You have simply too much SQL to change or regression-testing would be too prohibitive.

I’ll demonstrate the performance benefits of using SQL and offer some alternative techniques that you can use to refactor those critical SQL statements (or even consider as a standard for new applications).

Use SQL

In Listing 9-12, I ran a sales report that generated over 918,000 executions of a GET_RATE PL/SQL function. I demonstrated that a SQL-only implementation of the same report (i.e. a simple join to the RATES table) reduced the runtime of that query from 66 seconds to just 1 second (there was also a massive reduction in logical I/O). The SQL-only technique can also make a difference to the elapsed time of queries that use quicker PL/SQL functions, such as those that don’t contain any SQL.

SQL is an extremely flexible and rich language for processing sets of data quickly. Assuming that performance is your goal, you can use SQL such as analytic functions, subquery factoring, and a wide range of applications of the built-in functions to refactor your PL/SQL functions to SQL (visit the OTN SQL and PL/SQL forum or http://oraqa.com/ for examples of innovative uses for SQL).

Use a View

An argument in favour of retaining PL/SQL functions is that they encapsulate and centralize application logic, preventing a proliferation of the same rules throughout the code base. There are alternative encapsulation methods, such as using views, which have the added benefits of also providing the performance of SQL. In Listing 9-26, I’ve converted the sales report by calendar year and product from using the GET_RATE PL/SQL function to using a view, where the logic is encapsulated in a SQL expression.

Listing 9-26. Encapsulating Function Logic in a View

SQL> CREATE VIEW sales_rates_view
  2  AS
  3     SELECT s.*
  4     ,      s.amount_sold * (SELECT r.exchange_rate
  5                             FROM   rates r
  6                             WHERE  r.base_ccy   = 'USD'
  7                             AND    r.target_ccy = 'GBP'
  8                             AND    r.rate_date  = s.time_id) AS amount_sold_gbp
  9     FROM   sales     s;

View created.

With this view, I’ve converted the GET_RATE PL/SQL function to a scalar subquery lookup. The entire rates conversion is now projected as a single column. I’ve used a scalar subquery instead of a simple join to RATES to take advantage of an optimization with views known as column-elimination. With this optimization, the scalar subquery will only be executed when the AMOUNT_SOLD_GBP column is referenced in a query. This means that queries that don’t need the converted rate can also take advantage of this view without penalty, widening the scope for re-use. The timing and statistics for the sales report when using this view are shown in Listing 9-27.

Listing 9-27. Exploiting Column-Elimination in Views (No Column-Elimination)

SQL> SELECT t.calendar_year
  2  ,      p.prod_name
  3  ,      SUM(s.quantity_sold)   AS qty_sold
  4  ,      SUM(s.amount_sold)     AS amt_sold_usd
  5  ,      SUM(s.amount_sold_gbp) AS amt_sold_gbp
  6  FROM   sales_rates_view s
  7  ,      times            t
  8  ,      products         p
  9  WHERE  s.time_id    = t.time_id
10  AND    s.prod_id    = p.prod_id
 11  GROUP  BY
 12         t.calendar_year
 13  ,      p.prod_name;

272 rows selected.

Elapsed: 00:00:01.29

Execution Plan
----------------------------------------------------------
Plan hash value: 625253124

------------------------------------------------------------
| Id  | Operation                      | Name     | Rows  |
------------------------------------------------------------
|   0 | SELECT STATEMENT               |          |   252 |
|*  1 |  INDEX UNIQUE SCAN             | RATES_PK |     1 |
|   2 |  HASH GROUP BY                 |          |   252 |
|*  3 |   HASH JOIN                    |          |   918K|
|   4 |    PART JOIN FILTER CREATE     | :BF0000  |  1826 |
|   5 |     TABLE ACCESS FULL          | TIMES    |  1826 |
|*  6 |    HASH JOIN                   |          |   918K|
|   7 |     TABLE ACCESS FULL          | PRODUCTS |    72 |
|   8 |     PARTITION RANGE JOIN-FILTER|          |   918K|
|   9 |      TABLE ACCESS FULL         | SALES    |   918K|
------------------------------------------------------------

Statistics
----------------------------------------------------------
          8  recursive calls
          0  db block gets
      10639  consistent gets
          0  physical reads
          0  redo size
      17832  bytes sent via SQL*Net to client
        437  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        272  rows processed

I’ve highlighted the critical parts of the output. There is slightly more logical I/O than with a simple join to RATES, but this is minor compared with the I/O generated by the PL/SQL function. The sales report is still much quicker than the version with the PL/SQL function and the business logic is encapsulated.

Listing 9-28 demonstrates column-elimination. I’ve excluded the AMOUNT_SOLD_GBP column from the report and captured the statistics again.

Listing 9-28. Exploiting Column-Elimination in Views (Column-Elimination)

SQL> SELECT t.calendar_year
  2  ,      p.prod_name
  3  ,      SUM(s.quantity_sold)   AS qty_sold
  4  ,      SUM(s.amount_sold)     AS amt_sold_usd
  5  FROM   sales_rates_view s
  6  ,      times            t
  7  ,      products         p
  8  WHERE  s.time_id    = t.time_id
  9  AND    s.prod_id    = p.prod_id
 10  GROUP  BY
 11         t.calendar_year
 12  ,      p.prod_name;

272 rows selected.

Elapsed: 00:00:00.61

Statistics
----------------------------------------------------------
          8  recursive calls
          0  db block gets
       1736  consistent gets
          0  physical reads
          0  redo size
      11785  bytes sent via SQL*Net to client
        437  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        272  rows processed

This time, the query completes in half the time and the logical I/O decreases again. This is because the RATES table lookup has been eliminated from the view projection at runtime. The execution plan and projection in Listing 9-29 clearly demonstrate this.

Listing 9-29. Exploiting Column-Elimination in Views (Execution Plan)

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------
Plan hash value: 171093611

-----------------------------------------------------
| Id  | Operation                        | Name     |
-----------------------------------------------------
|   0 | SELECT STATEMENT                 |          |
|   1 |  HASH GROUP BY                   |          |
|   2 |   HASH JOIN                      |          |
|   3 |    TABLE ACCESS FULL             | PRODUCTS |
|   4 |    VIEW                          | VW_GBC_9 |
|   5 |     HASH GROUP BY                |          |
|   6 |      HASH JOIN                   |          |
|   7 |       PART JOIN FILTER CREATE    | :BF0000  |
|   8 |        TABLE ACCESS FULL         | TIMES    |
|   9 |       PARTITION RANGE JOIN-FILTER|          |
|  10 |        TABLE ACCESS FULL         | SALES    |
-----------------------------------------------------

Column Projection Information (identified by operation id):
-----------------------------------------------------------

   1 - "ITEM_4"[NUMBER,22], "P"."PROD_NAME"[VARCHAR2,50],
       SUM("ITEM_2")[22], SUM("ITEM_3")[22]
   2 - (#keys=1) "P"."PROD_NAME"[VARCHAR2,50], "ITEM_4"[NUMBER,22],
       "ITEM_2"[NUMBER,22], "ITEM_3"[NUMBER,22]
   3 - "P"."PROD_ID"[NUMBER,22], "P"."PROD_NAME"[VARCHAR2,50]
   4 - "ITEM_1"[NUMBER,22], "ITEM_2"[NUMBER,22], "ITEM_3"[NUMBER,22],
       "ITEM_4"[NUMBER,22]
   5 - "T"."CALENDAR_YEAR"[NUMBER,22], "S"."PROD_ID"[NUMBER,22],
       SUM("S"."AMOUNT_SOLD")[22], SUM("S"."QUANTITY_SOLD")[22]
   6 - (#keys=1) "T"."CALENDAR_YEAR"[NUMBER,22],
       "S"."PROD_ID"[NUMBER,22], "S"."AMOUNT_SOLD"[NUMBER,22],
       "S"."QUANTITY_SOLD"[NUMBER,22]
   7 - "T"."TIME_ID"[DATE,7], "T"."TIME_ID"[DATE,7],
       "T"."CALENDAR_YEAR"[NUMBER,22]
   8 - "T"."TIME_ID"[DATE,7], "T"."CALENDAR_YEAR"[NUMBER,22]
   9 - "S"."PROD_ID"[NUMBER,22], "S"."TIME_ID"[DATE,7],
       "S"."QUANTITY_SOLD"[NUMBER,22], "S"."AMOUNT_SOLD"[NUMBER,22]
  10 - "S"."PROD_ID"[NUMBER,22], "S"."TIME_ID"[DATE,7],
       "S"."QUANTITY_SOLD"[NUMBER,22], "S"."AMOUNT_SOLD"[NUMBER,22]

This highlights the fact that views can be a higher-performing alternative to PL/SQL functions, without compromising the need for encapsulation and centralization. As a final consideration, you could rename the base table and allow the view to take on its name. With these actions, you will have “slotted in” the business logic columns into the SALES “table” and your DML statements will not be compromised, as Listing 9-30 shows.

Listing 9-30. DML on a Table through a View with a Scalar Subquery

SQL> RENAME sales TO sales_t;

Table renamed.

SQL> CREATE VIEW sales
  2  AS
  3     SELECT s. prod_id, s.cust_id, s.time_id, s.channel_id,
  4            s.promo_id, s.quantity_sold, s.amount_sold,
  5            s.amount_sold * (SELECT r.exchange_rate
  6                             FROM   rates r
  7                             WHERE  r.base_ccy   = 'USD'
  8                             AND    r.target_ccy = 'GBP'
  9                             AND    r.rate_date  = s.time_id) AS amount_sold_gbp
 10     FROM   sales_t  s;
View created.

SQL> INSERT INTO sales
  2     ( prod_id, cust_id, time_id, channel_id, promo_id,
  3       quantity_sold, amount_sold )
  4  VALUES
  5     ( 13, 987, DATE '1998-01-10', 3, 999, 1, 1232.16 );

1 row created.

SQL> UPDATE sales
  2  SET    amount_sold = 10000
  3  WHERE  ROWNUM = 1;

1 row updated.

SQL> DELETE
  2  FROM   sales
  3  WHERE  amount_sold = 10000;

1 row deleted.

To use this approach, consideration needs to be given to any existing code base, such as %ROWTYPE declarations, PL/SQL record-based DML, unqualified INSERT or SELECT lists (i.e. those without explicit column references), and so on. Fortunately, derived columns in views cannot be modified, as the final example for this section demonstrates in Listing 9-31.

Listing 9-31. DML on a Table through a View with a Scalar Subquery

SQL> INSERT INTO sales
  2     ( prod_id, cust_id, time_id, channel_id, promo_id,
  3       quantity_sold, amount_sold, amount_sold_gbp )
  4  VALUES
  5     ( 13, 987, DATE '1998-01-10', 3, 999, 1, 1232.16, 1000 );
     quantity_sold, amount_sold, amount_sold_gbp )
                                 *
ERROR at line 3:
ORA-01733: virtual column not allowed here
Use a Virtual Column

To wrap up this section on using SQL expressions in place of PL/SQL functions, I’m going to demonstrate how to replace a PL/SQL function with a virtual column.

A feature of Oracle Database 11g, virtual columns are expressions stored as column metadata in tables (they don’t use any storage as such). They are logically similar to columns in views but are far more flexible as they can have statistics gathered for them or be indexed. Virtual columns are an excellent tool for encapsulating simple business rules and, because they are stored as metadata against tables, they are also self-documenting declarations of your data logic.

To demonstrate virtual columns as an alternative to PL/SQL functions, I’ve converted the sales report from Listing 9-6. To recap, this used the FORMAT_CUSTOMER_NAME function to format a customer name and took almost 7 seconds to execute (compared with just 1 second as a SQL-only query). To begin the conversion, I’ve added a virtual column to the CUSTOMERS table, as shown in Listing 9-32.

Listing 9-32. Converting a PL/SQL Function to a Virtual Column (Syntax)

SQL> ALTER TABLE customers ADD
  2  ( cust_name VARCHAR2(100) GENERATED ALWAYS AS (cust_first_name||' '||cust_last_name) )
  3  ;

Table altered.

The GENERATED ALWAYS AS (expression) syntax is specific to virtual columns (I’ve omitted an optional VIRTUAL keyword to keep the display on a single line). The CUSTOMERS table now looks as follows in Listing 9-33.

Listing 9-33. Converting a PL/SQL Function to a Virtual Column (Table Description)

SQL> SELECT column_name
  2  ,      data_type
  3  ,      data_default
  4  FROM   user_tab_columns
  5  WHERE  table_name = 'CUSTOMERS'
  6  ORDER  BY
  7         column_id;

COLUMN_NAME                    DATA_TYPE       DATA_DEFAULT
------------------------------ --------------- -----------------------------------------
CUST_ID                        NUMBER
CUST_FIRST_NAME                VARCHAR2
CUST_LAST_NAME                 VARCHAR2
CUST_GENDER                    CHAR
<...output removed...>
CUST_VALID                     VARCHAR2
CUST_NAME                      VARCHAR2        "CUST_FIRST_NAME"||' '||"CUST_LAST_NAME

Using the virtual column in a SQL statement is no different from any other table, view, or other column. The converted sales report is shown in Listing 9-34.

Listing 9-34. Converting a PL/SQL Function to a Virtual Column (Usage)

SQL> SELECT t.calendar_year
  2  ,      c.cust_name
  3  ,      SUM(s.quantity_sold) AS qty_sold
  4  ,      SUM(s.amount_sold)   AS amt_sold
  5  FROM   sales     s
  6  ,      customers c
  7  ,      times     t
  8  WHERE  s.cust_id = c.cust_id
  9  AND    s.time_id = t.time_id
 10  GROUP  BY
11         t.calendar_year
 12  ,      c.cust_name;

11604 rows selected.

Elapsed: 00:00:01.20

As you can see from the elapsed time of this query, by moving the customer name formatting to a virtual column, I’ve eliminated the overhead of the PL/SQL function while retaining the encapsulation of the business logic with the very data to which it applies.

Reducing Executions

Earlier in this chapter, I described the fact that you can’t reliably predict the number calls to a PL/SQL function that a SQL statement will make. Further, each execution carries a context-switching penalty in addition to the work the PL/SQL function has to do. There are, however, techniques you can use to reliably reduce the function calls and I will demonstrate some of these next.

Pre/Post-Computing with SQL Hints

I gave an example of the CBO’s cost-based query transformation in Listings 9-8 to 9-10, where my attempt to pre-compute formatted customer names in an inline view was negated by view merging. The result was 918,000 PL/SQL function executions (i.e. once per row in the larger SALES table) instead of an intended 55,500 calls (i.e. once per row in the CUSTOMERS table). For some queries, such an order of magnitude or more function executions can be disastrous.

I’ve repeated the customer sales report in Listing 9-35. This time, however, I’ve used a couple of hints to ensure that the pre-computation of the customer names in an inline view “sticks” and the CBO doesn’t merge the query blocks.

Listing 9-35. Reducing Function Calls through SQL

SQL> SELECT /*+ NO_MERGE(@customers) */
  2         t.calendar_year
  3  ,      c.cust_name
  4  ,      SUM(s.quantity_sold) AS qty_sold
  5  ,      SUM(s.amount_sold)   AS amt_sold
  6  FROM   sales     s
  7  ,     (
  8         SELECT /*+ QB_NAME(customers) */
  9                cust_id
 10         ,      format_customer_name (
 11                   cust_first_name, cust_last_name
 12                   ) AS cust_name
 13         FROM   customers
 14        )          c
 15  ,      times     t
 16  WHERE  s.cust_id = c.cust_id
 17  AND    s.time_id = t.time_id
 18  GROUP  BY
19         t.calendar_year
 20  ,      c.cust_name
 21  ;

11604 rows selected.

Elapsed: 00:00:01.49

SQL> exec counter.show('Function calls'),

Function calls: 55500

You can see that this time, I’ve used hints to instruct the CBO to leave my inline view unmerged. The aim is to ensure that my PL/SQL function only executes against the CUSTOMERS dataset and not the larger SALES set. I’ve done this by using the NO_MERGE hint, adopting the query block naming syntax introduced in Oracle Database 10g. The QB_NAME hint is used to label the inline view query block; I’m therefore able to reference the inline view from the main query block, as you can see in the NO_MERGE hint.

I can go one step further with this query, however. In this particular example, the FORMAT_CUSTOMER_NAME function only needs to be applied to the final resultset of just11,604 rows (far fewer than the 55,500 customer records). I can therefore pre-group the entire resultset using the same hints as previously, but only call the PL/SQL function at the very last stage. Listing 9-36 shows the impact of this.

Listing 9-36. Reducing Function Calls through SQL (Restructured Query)

SQL> SELECT /*+ NO_MERGE(@pregroup) */
  2         calendar_year
  3  ,      format_customer_name(
  4            cust_first_name, cust_last_name
  5            ) AS cust_name
  6  ,      qty_sold
  7  ,      amt_sold
  8  FROM  (
  9           SELECT /*+ QB_NAME(pregroup) */
 10                  t.calendar_year
 11           ,      c.cust_first_name
 12           ,      c.cust_last_name
 13           ,      SUM(s.quantity_sold) AS qty_sold
 14           ,      SUM(s.amount_sold)   AS amt_sold
 15           FROM   sales     s
 16           ,      customers c
 17           ,      times     t
 18           WHERE  s.cust_id = c.cust_id
 19           AND    s.time_id = t.time_id
 20           GROUP  BY
 21                  t.calendar_year
 22           ,      c.cust_first_name
 23           ,      c.cust_last_name
 24        );
11604 rows selected.

Elapsed: 00:00:01.14

SQL> exec counter.show('Function calls'),

Function calls: 11604

This is better still, because I have managed to pre-aggregate all the data prior to calling the PL/SQL function, reducing its number of calls to 11,604 and saving more time on the report.

This technique can be applied in targeted cases where PL/SQL function calls absolutely must be kept to a minimum. For example, by adopting this technique for the earlier sales report with the GET_RATE function, I managed to reduce 918,000 function executions to just 36,000, which decreased the overall response time from 66 seconds to just 4 seconds.

A more strategic method for reducing function calls, however, is to exploit a caching solution, which I’ll now describe.

Caching to Reduce Function Calls

There are several options for caching data in Oracle Database, depending on your version. I’m going to briefly describe two caching techniques for reducing function calls that you might wish to investigate and/or apply to your application. These are:

  • Scalar-subquery caching
  • Cross-session PL/SQL function result cache
Scalar-Subquery Caching

This caching feature is an internal optimization designed to reduce the number of executions of SQL statements or PL/SQL functions embedded in scalar subqueries. Although the efficiency of this internal cache can’t be reliably predicted, it can be used to reduce PL/SQL function calls to some extent. In Listing 9-37, I’ve converted the sales report with the GET_RATES lookup function to exploit the scalar subquery cache.

Listing 9-37. Using Scalar-Subquery Caching to Reduce PL/SQL Function Calls

SQL> SELECT t.calendar_year
  2  ,      p.prod_name
  3  ,      SUM(s.amount_sold)                                             AS amt_sold_usd
  4  ,      SUM(s.amount_sold * (SELECT get_rate(s.time_id, 'USD', 'GBP')
  5                              FROM   dual))                             AS amt_sold_gbp
  6  FROM   sales     s
  7  ,      products  p
  8  ,      times     t
  9  WHERE  s.prod_id = p.prod_id
 10  AND    s.time_id = t.time_id
 11  GROUP  BY
12         t.calendar_year
 13  ,      p.prod_name
 14  ;

272 rows selected.

Elapsed: 00:00:02.54

Statistics
----------------------------------------------------------
      19451  recursive calls
          0  db block gets
      40634  consistent gets
          0  physical reads
          0  redo size
      16688  bytes sent via SQL*Net to client
        437  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        272  rows processed

SQL> exec counter.show('Function calls'),

Function calls: 19450

By encapsulating the call to GET_RATE in a scalar subquery, I’ve managed to reduce the query’s elapsed time from 66 seconds to just under 3 seconds. The counter shows that the PL/SQL function calls have dropped to just 19,450 (from over 918,000), resulting in an enormous saving in time and logical I/O (over 1.8 million consistent gets were prevented by the caching).

As noted earlier, the cache’s efficiency is dependent on a range of factors, including the order of the input data. With this in mind, I know that the range of input-values to the GET_RATE function in my SALES data is quite small, so in Listing 9-38, I have attempted to reduce the function calls further still by ordering the inputs into the scalar subquery.

Listing 9-38. Using Scalar Subquery Caching to Reduce PL/SQL Function Calls (Effect of Sorted Data)

SQL> SELECT /*+ NO_MERGE(@inner) */
  2         calendar_year
  3  ,      prod_name
  4  ,      SUM(amount_sold)                                          AS amt_sold_usd
  5  ,      SUM(amount_sold * (SELECT get_rate(time_id, 'USD', 'GBP')
  6                            FROM   dual))                          AS amt_sold_gbp
  7  FROM  (
  8         SELECT /*+ QB_NAME(inner) NO_ELIMINATE_OBY */
  9                t.calendar_year
 10         ,      s.time_id
 11         ,      p.prod_name
 12         ,      s.amount_sold
 13         FROM   sales     s
 14         ,      products  p
 15         ,      times     t
16         WHERE  s.prod_id = p.prod_id
 17         AND    s.time_id = t.time_id
 18         ORDER  BY
 19                s.time_id
 20        )
 21  GROUP  BY
 22         calendar_year
 23  ,      prod_name
 24  ;

272 rows selected.

Elapsed: 00:00:02.14

Statistics
----------------------------------------------------------
       1461  recursive calls
          0  db block gets
       4662  consistent gets
          8  physical reads
          0  redo size
      16688  bytes sent via SQL*Net to client
        437  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          2  sorts (memory)
          0  sorts (disk)
        272  rows processed

SQL> exec counter.show('Function calls: ordered inputs'),

Function calls: ordered inputs: 1460

By ordering the inputs to the scalar subquery cache, I’ve reduced the PL/SQL function calls further to just 1,460 (and therefore reduced the logical I/O by an order of magnitude and trimmed the overall elapsed time by a small margin). This demonstrates that ordering (or clustering) of calls into the subquery cache can make a difference to its efficiency. In my case, the savings I made through the further reduction in PL/SQL function calls “paid for” the cost of pre-sorting the SALES data.

Cross-Session PL/SQL Function Result Cache

The cross-session PL/SQL function Result Cache was introduced in Oracle Database 11g. The principle behind result caching is simple. First, a PL/SQL function is flagged for caching (using the RESULT_CACHE directive). Thereafter, each time the function is invoked with new parameters, Oracle Database executes the function, adds the return value(s) to the Result Cache, and returns the result(s) to the calling context. If a call is repeated, Oracle Database retrieves the results from the cache rather than re-execute the function. Under certain circumstances, this caching behavior can result in significant performance gains. It also has the benefit of requiring very little re-factoring (just a small function change, in fact).

I’ll demonstrate the effect of the Result Cache on the GET_RATE function. First, I need to re-compile the function with the cache directive, as shown in Listing 9-39.

Listing 9-39. Preparing a Function for the PL/SQL Function Result Cache

SQL> CREATE OR REPLACE FUNCTION get_rate(
  2                             p_rate_date IN rates.rate_date%TYPE,
  3                             p_from_ccy  IN rates.base_ccy%TYPE,
  4                             p_to_ccy    IN rates.target_ccy%TYPE
  5                             ) RETURN rates.exchange_rate%TYPE
  6                               RESULT_CACHE RELIES_ON (rates) AS
  7     v_rate rates.exchange_rate%TYPE;
  8  BEGIN
     <...snip...>
 15  END get_rate;
 16  /

Function created.

I’ve added two pieces of syntax specific to the Result Cache, as highlighted above. The RESULT_CACHE keyword is self-explanatory and tells Oracle Database to enable this function for the Result Cache. The RELIES_ON (rates) syntax is required only in Oracle Database 11g Release 1 and declares the fact that the function is dependent on the data in the RATES table (Oracle Database 11g Release 2 automatically recognizes such dependencies). Therefore, any transaction against RATES will cause Oracle Database to invalidate the cached results and suspend caching this function until the transaction is complete (at which point the cache will commence a new refresh cycle).

Now that I have cache-enabled this function, I can re-execute my original sales query, as demonstrated in Listing 9-40.

Listing 9-40. Query Performance with a Result-Cached PL/SQL Function

SQL> SELECT t.calendar_year
  2  ,      p.prod_name
  3  ,      SUM(s.amount_sold)                     AS amt_sold_usd
  4  ,      SUM(s.amount_sold *
  5             get_rate(s.time_id, 'USD', 'GBP')) AS amt_sold_gbp
  6  FROM   sales     s
  7  ,      products  p
  8  ,      times     t
  9  WHERE  s.prod_id = p.prod_id
 10  AND    s.time_id = t.time_id
 11  GROUP  BY
 12         t.calendar_year
 13  ,      p.prod_name
 14  ;

272 rows selected.

Elapsed: 00:00:08.74

Statistics
----------------------------------------------------------
       1481  recursive calls
          0  db block gets
4683  consistent gets
          0  physical reads
          0  redo size
      16688  bytes sent via SQL*Net to client
        437  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        272  rows processed

By caching the function results, the elapsed time of this query has dropped by 57 seconds. This isn’t quite as quick as the SQL-only implementation (that completed in 1 second) or the scalar subquery version (that completed in 2 seconds), but you might consider this to be an acceptable performance gain, given that no SQL needed to be re-written to achieve it and its behavior is predictable.

Another issue with the original query was the I/O generated by the encapsulated RATES lookup. With the Result Cache, the logical I/O has reduced to a more acceptable level.

To quantify the reduction in function executions, I traced the query with the PL/SQL Hierarchical Profiler. The profile report is provided in Listing 9-41.

Listing 9-41. PL/SQL Hierarchical Profiler Session Report for a Query with a Result-Cached PL/SQL Function

FUNCTION                                  LINE#      CALLS SUB_ELA_US FUNC_ELA_US
------------------------------------ ---------- ---------- ---------- -----------
__plsql_vm                                    0     918846    3777356     3651587
GET_RATE                                      1       1460     125433       25731
GET_RATE.__static_sql_exec_line9              9       1460      99702       99702
__anonymous_block                             0          3        336         276
DBMS_OUTPUT.GET_LINES                       180          2         60          57
DBMS_OUTPUT.GET_LINE                        129          2          3           3
DBMS_HPROF.STOP_PROFILING                    59          1          0           0

You can see that the cache-enabled GET_RATES function was executed just 1,460 times (the same result as the best scalar subquery cache example). Interestingly, context-switching into the PL/SQL VM is not reduced by using the Result Cache, so Oracle Database therefore passed control to the PL/SQL engine once for each of the 918,000 rows the query needed to process. For this reason, it rarely pays to convert a PL/SQL-only function (i.e. one without any embedded SQL) to use the Result Cache. First, there is no reduction in context-switching and second, you’ll possibly find that diverting a function call to the Result Cache and acquiring the RC latch that protects it will take longer than an execution of the function itself.

To wrap up, Oracle Database also provides several performance views for the Result Cache. For example, the V$RESULT_CACHE_STATISTICS view can provide a useful overview of the cache’s efficiency. Listing 9-40 ran with an empty cache, so the statistics in Listing 9-42 are entirely attributed to this query.

Listing 9-42. Investigating Result Cache Hits with V$RESULT_CACHE_STATISTICS

SQL> SELECT name, value
  2  FROM   v$result_cache_statistics
  3  WHERE  name IN ('Find Count','Create Count Success'),
NAME                    VALUE
----------------------- ----------
Create Count Success    1460
Find Count              917383

These statistics demonstrate a very efficient cache. The GET_RATE function was executed just 1,460 times to load the cache (as demonstrated by the Create Count Success statistic, which also corresponds to the GET_RATE statistics in the Hierarchical Profiler report in Listing 9-41). Once cached, the results were heavily reused (the Find Count statistic shows that the cached results were used over 917,000 times). If I were to repeat the sales query at this point, the PL/SQL function would not be executed at all (assuming the cached results were not invalidated) and the response time would be further reduced, albeit by a small margin.

images Note If you’re not running Oracle Database 11g, you can simulate the Result Cache (to a degree) by creating your own session-based PL/SQL cache with associative arrays. While this will not reduce function executions in the same way, it will dramatically reduce the impact of the SQL lookups embedded inside the function. See the “Tuning PL/SQL” section at the end of this chapter for an example that converts the GET_RATE function from the result-cached version to a user-defined array-cached implementation. However, make sure that you consider the potential shortcomings of such an approach, as described in that section.

Deterministic Functions

As stated earlier, Oracle Database has an internal optimization for deterministic functions which can sometimes make them a useful feature for reducing PL/SQL function calls. In Listing 9-43, I’ve demonstrated the effect of declaring the GET_CUSTOMER_NAME function to be deterministic (i.e. by using the DETERMINISTIC keyword in the function specification) and running a variation on the sales-by-customer report from Listing 9-5.

Listing 9-43. Effect of Declaring a DETERMINISTIC Function

SQL> CREATE OR REPLACE FUNCTION format_customer_name (
  2                             p_first_name IN VARCHAR2,
  3                             p_last_name  IN VARCHAR2
  4                             ) RETURN VARCHAR2 DETERMINISTIC AS
<snip>
  9 /

Function created.

SQL> SELECT /*+ NO_MERGE(@inner) */
  2         calendar_year
  3  ,      format_customer_name(
  4            cust_first_name, cust_last_name
  5            )                                AS cust_name
6  ,      SUM(quantity_sold)                  AS qty_sold
  7  ,      SUM(amount_sold)                    AS amt_sold
  8  FROM  (
  9         SELECT /*+
 10                    QB_NAME(inner)
 11                    NO_ELIMINATE_OBY
 12                 */
 13                t.calendar_year
 14         ,      c.cust_first_name
 15         ,      c.cust_last_name
 16         ,      s.quantity_sold
 17         ,      s.amount_sold
 18         FROM   sales     s
 19         ,      customers c
 20         ,      times     t
 21         WHERE  s.cust_id = c.cust_id
 22         AND    s.time_id = t.time_id
 23         ORDER  BY
 24                c.cust_first_name
 25         ,      c.cust_last_name
 26        )
 27  GROUP  BY
 28         calendar_year
 29  ,      format_customer_name(
 30            cust_first_name, cust_last_name
 31            )
 32  ;

11604 rows selected.

Elapsed: 00:00:07.83

Statistics
----------------------------------------------------------
       3189  consistent gets
          0  physical reads

SQL> exec counter.show('Deterministic function calls'),

Deterministic function calls: 912708

You can see from the results that the number of PL/SQL function calls has reduced by a very small margin (approximately 6,000 fewer than the original query in Listing 9-5) so it hasn’t been particularly effective (this report is actually slower than the original). In fact, to benefit from the deterministic function optimization at all with this example, I needed to pre-order the input data to ensure that customer names passing into the function were clustered together. In this case, the cost of sorting the data outweighed the marginal gains from 6,000 fewer calls to FORMAT_CUSTOMER_NAME and my report ran slower as a result.

That is not to say that the optimization for deterministic functions doesn’t work. As noted earlier, there are a range of factors that affect its efficiency. It can be quite effective in some cases, particularly when your resultset contains very few distinct function inputs. For example, there are approximately 3,700 distinct customers in the SALES table, so I simplified my sales query to join SALES to CUSTOMERS, order the data in an in-line view and call the FORMAT_CUSTOMER_NAME function with no aggregation. With this simplified query, the PL/SQL function was executed just 36,000 times for 918,000 rows. This is better than the standard sales report, of course, albeit quite a restricted optimization when compared with the alternative techniques I’ve demonstrated.

images Caution By their very nature, PL/SQL functions that return randomized data or perform SQL lookups are not deterministic. Don’t be tempted to declare a function as deterministic unless it really is or you will risk returning wrong results from your queries.

Assisting the CBO

As described earlier, there are several ways in which PL/SQL functions can trip up the Cost-Based Optimizer, largely due to the lack of statistics and the reliance on defaults. There are a range of methods you can use to improve the CBO’s handling SQL with PL/SQL functions and I’ll demonstrate the following techniques:

  • Function-based indexes
  • Extended statistics
  • Default statistics
  • The Extensible Optimizer
Function-Based Indexes

Functions applied to an indexed column completely disable the index for use in an execution plan. To counter this, function-based indexes have been available since Oracle 8i and allow such expressions to be indexed. In the following listings, I’ve demonstrated the use of a function-based index on a query that uses a PL/SQL function predicate. Listing 9-44 begins with a simple PL/SQL function applied to a highly–selective indexed column.

Listing 9-44. PL/SQL Function Applied to Indexed Column

SQL> CREATE FUNCTION promo_function(
  2                  p_promo_category IN VARCHAR2
  3                  ) RETURN VARCHAR2 DETERMINISTIC IS
  4  BEGIN
  5     RETURN UPPER(p_promo_category);
  6  END promo_function;
  7  /

Function created.
SQL> SELECT *
  2  FROM   sales       s
  3  ,      promotions  p
  4  ,      times       t
  5  WHERE  s.promo_id = p.promo_id
  6  AND    s.time_id  = t.time_id
  7  AND    t.time_id BETWEEN DATE '2000-01-01' AND DATE '2000-03-31'
  8  AND    promo_function(p.promo_category) = 'AD NEWS';

Execution Plan
----------------------------------------------------------

-------------------------------------------------------------
| Id  | Operation                     | Name       | Rows  |
-------------------------------------------------------------
|   0 | SELECT STATEMENT              |            | 51609 |
|*  1 |  HASH JOIN                    |            | 51609 |
|   2 |   PART JOIN FILTER CREATE     | :BF0000    |    92 |
|   3 |    TABLE ACCESS BY INDEX ROWID| TIMES      |    92 |
|*  4 |     INDEX RANGE SCAN          | TIMES_PK   |    92 |
|*  5 |   HASH JOIN                   |            | 52142 |
|*  6 |    TABLE ACCESS FULL          | PROMOTIONS |     5 |
|   7 |    PARTITION RANGE SINGLE     |            | 62197 |
|*  8 |     TABLE ACCESS FULL         | SALES      | 62197 |
-------------------------------------------------------------

You can see that despite having an index on the PROMO_CATEGORY column, applying the PL/SQL function to the column forces the CBO to choose a full table scan of PROMOTIONS (it has no other choice). Fortunately, I can use a function-based index to work around this (only because PROMO_FUNCTION is deterministic). In Listing 9-45, I’ve created the function-based index and included the new execution plan.

Listing 9-45. Using a Function-Based Index

SQL> CREATE INDEX promotions_fbi
  2     ON promotions (promo_function(promo_category))
  3     COMPUTE STATISTICS;

Index created.

Execution Plan
----------------------------------------------------------
Plan hash value: 3568243509

-----------------------------------------------------------------
| Id  | Operation                     | Name           | Rows  |
-----------------------------------------------------------------
|   0 | SELECT STATEMENT              |                | 51609 |
|*  1 |  HASH JOIN                    |                | 51609 |
|   2 |   PART JOIN FILTER CREATE     | :BF0000        |    92 |
|   3 |    TABLE ACCESS BY INDEX ROWID| TIMES          |    92 |
|*  4 |     INDEX RANGE SCAN          | TIMES_PK       |    92 |
|*  5 |   HASH JOIN                   |                | 52142 |
|   6 |    TABLE ACCESS BY INDEX ROWID| PROMOTIONS     |     5 |
|*  7 |     INDEX RANGE SCAN          | PROMOTIONS_FBI |     2 |
|   8 |    PARTITION RANGE SINGLE     |                | 62197 |
|*  9 |     TABLE ACCESS FULL         | SALES          | 62197 |
-----------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - access("S"."TIME_ID"="T"."TIME_ID")
   4 - access("T"."TIME_ID">=TO_DATE(' 2000-01-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND
              "T"."TIME_ID"<=TO_DATE(' 2000-03-31 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
   5 - access("S"."PROMO_ID"="P"."PROMO_ID")
   7 - access("SH"."PROMO_FUNCTION"("PROMO_CATEGORY")='AD NEWS')
   9 - filter("S"."TIME_ID"<=TO_DATE(' 2000-03-31 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))

This time the CBO has chosen to use the function-based index. Not only does the index exist, but the CBO also has some statistics to work with in choosing an optimal plan.

If you wish to investigate function-based indexes, you will find metadata regarding their implementation in views such as USER_INDEXES, USER_IND_EXPRESSIONS, and USER_TAB_COLS/USER_IND_COLUMNS (the indexed expression is stored as a hidden virtual column in the table being indexed).

Extended Statistics

Statistics are one of the benefits of function-based indexes. These can be invaluable to the CBO (even if it chooses not to use the index, it can make use of the statistics in deciding join or predicate orders). However, indexing a PL/SQL function predicate might not always be possible. In such cases, Oracle Database 11g provides an alternative known as extended statistics.

Extended statistics can be created on expressions without requiring a supporting index. Such statistics are known as extensions and they have an added benefit over function-based indexes of being applicable to all PL/SQL function expressions in a query (i.e. not just those in predicates).

To demonstrate this, I have re-executed the sales report by customer with extended statistics in place. Listing 9-46 is a reminder of the query without extended statistics, together with its execution plan and function call output from Listing 9-5.

Listing 9-46. Using Extended Statistics on a Projected PL/SQL Function (Plan without Statistics from Listing 9-5)

SQL> SELECT t.calendar_year
  2  ,      format_customer_name(
  3            c.cust_first_name, c.cust_last_name
  4            )                 AS cust_name
  5  ,      SUM(s.quantity_sold) AS qty_sold
  6  ,      SUM(s.amount_sold)   AS amt_sold
  7  FROM   sales     s
  8  ,      customers c
  9  ,      times     t
 10  WHERE  s.cust_id = c.cust_id
11  AND    s.time_id = t.time_id
 12  GROUP  BY
 13         t.calendar_year
 14  ,      format_customer_name(
 15            c.cust_first_name, c.cust_last_name
 16            )
 17  ;

11604 rows selected.

Elapsed: 00:00:06.94

Execution Plan
----------------------------------------------------------
Plan hash value: 3113689673

-------------------------------------------------------------
| Id  | Operation                      | Name      | Rows  |
-------------------------------------------------------------
|   0 | SELECT STATEMENT               |           |   918K|
|   1 |  HASH GROUP BY                 |           |   918K|
|*  2 |   HASH JOIN                    |           |   918K|
|   3 |    PART JOIN FILTER CREATE     | :BF0000   |  1826 |
|   4 |     TABLE ACCESS FULL          | TIMES     |  1826 |
|*  5 |    HASH JOIN                   |           |   918K|
|   6 |     TABLE ACCESS FULL          | CUSTOMERS | 55500 |
|   7 |     PARTITION RANGE JOIN-FILTER|           |   918K|
|   8 |      TABLE ACCESS FULL         | SALES     |   918K|
------------------------------------------------------------

Statistics
----------------------------------------------------------
       3189  consistent gets
          0  physical reads

SQL> exec counter.show('Function calls'),

Function calls: 918843

PL/SQL procedure successfully completed.

As you can see, this performs over 918,000 function calls and completes in roughly 7 seconds. In Listing 9-47, I’ve used DBMS_STATS to generate extended statistics on the PL/SQL function call and show the execution plan from the re-executed query.

Listing 9-47. Generating and Using Extended Statistics

SQL> BEGIN
  2     DBMS_STATS.GATHER_TABLE_STATS(
  3        ownname    => USER,
  4        tabname    => 'CUSTOMERS',
5        method_opt => 'FOR COLUMNS (format_customer_name(cust_first_name,cust_last_name))
SIZE AUTO'
  6        );
  7  END;
  8  /

PL/SQL procedure successfully completed.

SQL> SELECT t.calendar_year...

11604 rows selected.

Elapsed: 00:00:00.90

Execution Plan
----------------------------------------------------------
Plan hash value: 833790846

---------------------------------------------------------------
| Id  | Operation                        | Name      | Rows  |
---------------------------------------------------------------
|   0 | SELECT STATEMENT                 |           | 19803 |
|   1 |  HASH GROUP BY                   |           | 19803 |
|*  2 |   HASH JOIN                      |           | 24958 |
|   3 |    VIEW                          | VW_GBC_9  | 24958 |
|   4 |     HASH GROUP BY                |           | 24958 |
|*  5 |      HASH JOIN                   |           |   918K|
|   6 |       PART JOIN FILTER CREATE    | :BF0000   |  1826 |
|   7 |        TABLE ACCESS FULL         | TIMES     |  1826 |
|   8 |       PARTITION RANGE JOIN-FILTER|           |   918K|
|   9 |        TABLE ACCESS FULL         | SALES     |   918K|
|  10 |    TABLE ACCESS FULL             | CUSTOMERS | 55500 |
---------------------------------------------------------------

SQL> exec counter.show('Extended stats'),

Extended stats: 237917

PL/SQL procedure successfully completed.

With the benefit of the extended statistics for the PL/SQL function, the CBO has made an informed decision to rewrite the query and pre-group the SALES and TIMES data. This is represented by Step 3 of the execution plan. Because this pre-groups and therefore reduces the resultset, the PL/SQL function calls are reduced to below 238,000. Overall, the effect on the response time is dramatic.

Extended statistics are a good alternative to the NO_MERGE hint demonstrated earlier. Without extended statistics, the query shown in Listing 9-48 was merged into the main query block by the CBO, resulting in 918,000 function executions. Fortunately, generating the extended statistics works nicely to prevent this transformation, again with good results.

Listing 9-48. Extended Statistics and Impact on Cost-Based Query Transformation

SQL> SELECT t.calendar_year
  2  ,      c.cust_name
  3  ,      SUM(s.quantity_sold) AS qty_sold
  4  ,      SUM(s.amount_sold)   AS amt_sold
  5  FROM   sales     s
  6  ,     (
  7         SELECT cust_id
  8         ,      format_customer_name (
  9                   cust_first_name, cust_last_name
 10                   ) AS cust_name
 11         FROM   customers
 12        )          c
 13  ,      times     t
 14  WHERE  s.cust_id = c.cust_id
 15  AND    s.time_id = t.time_id
 16  GROUP  BY
 17         t.calendar_year
 18  ,      c.cust_name
 19  ;

11604 rows selected.

Elapsed: 00:00:00.94

Execution Plan
----------------------------------------------------------
Plan hash value: 833790846

---------------------------------------------------------------
| Id  | Operation                        | Name      | Rows  |
---------------------------------------------------------------
|   0 | SELECT STATEMENT                 |           | 19803 |
|   1 |  HASH GROUP BY                   |           | 19803 |
|*  2 |   HASH JOIN                      |           | 24958 |
|   3 |    VIEW                          | VW_GBC_9  | 24958 |
|   4 |     HASH GROUP BY                |           | 24958 |
|*  5 |      HASH JOIN                   |           |   918K|
|   6 |       PART JOIN FILTER CREATE    | :BF0000   |  1826 |
|   7 |        TABLE ACCESS FULL         | TIMES     |  1826 |
|   8 |       PARTITION RANGE JOIN-FILTER|           |   918K|
|   9 |        TABLE ACCESS FULL         | SALES     |   918K|
|  10 |    TABLE ACCESS FULL             | CUSTOMERS | 55500 |
---------------------------------------------------------------

SQL> exec counter.show('Extended stats + inline view'),

Extended stats + inline view: 17897

Interestingly, the CBO has generated the same execution plan for this re-arranged query, although the function executions have dropped dramatically from 234,000 to just 18,000. This is another excellent example of the function executions being unpredictable—I have the same execution plan yet more than an order of magnitude difference in function calls. Nevertheless, these listings clearly highlight how important statistics on PL/SQL functions are for the CBO and that extended statistics are an excellent mechanism for supplying them.

images Note Extended statistics, or extensions, are implemented using system-generated virtual columns (similar to function-based indexes). To investigate more about this feature, query dictionary views such as USER_TAB_COLS and USER_STAT_EXTENSIONS.

Default Statistics

Unlike extended statistics, default statistics are user-generated and only apply to PL/SQL functions in predicates. Furthermore, the CBO will not use default statistics to inform cost-based query transformations, but it will use them to determine predicate ordering in the event that you have more than one function call in your WHERE clause.

Default statistics are supplied using the ASSOCIATE STATISTICS SQL command and with these you can define statistics on selectivity, CPU, and I/O costs for your PL/SQL functions (thereby improving on the defaults that the CBO otherwise adopts).

In Listing 9-19, I highlighted the impact of default predicate ordering when I referenced two functions in a query (these were appropriately named QUICK_FUNCTION and SLOW_FUNCTION). Listing 9-49 demonstrates how applying default statistics can ensure that the CBO applies the function predicates in the most efficient order.

Listing 9-49. Setting Default Statistics for PL/SQL Function Predicates

SQL> ASSOCIATE STATISTICS WITH FUNCTIONS quick_function DEFAULT SELECTIVITY 0.1;

Statistics associated.

SQL> ASSOCIATE STATISTICS WITH FUNCTIONS slow_function DEFAULT SELECTIVITY 50;

Statistics associated.

SQL> SELECT *
  2  FROM   thousand_rows
  3  WHERE  slow_function(n1) = 0
  4  AND    quick_function(n1) = 0;

1 row selected.

Elapsed: 00:00:00.02
Execution Plan
----------------------------------------------------------
Plan hash value: 4012467810

-----------------------------------------------------------------------------------
| Id  | Operation         | Name          | Rows  | Bytes | Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |               |     1 |    55 |     5   (0)| 00:00:01 |
|*  1 |  TABLE ACCESS FULL| THOUSAND_ROWS |     1 |    55 |     5   (0)| 00:00:01 |
-----------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("QUICK_FUNCTION"("N1")=0 AND "SLOW_FUNCTION"("N1")=0)

Because of the selectivity statistics associated with SLOW_FUNCTION and QUICK_FUNCTION, you can see that the CBO has chosen to re-order the predicates to good effect. I have told Oracle Database that an equality predicate against QUICK_FUNCTION will only be true 1 in 1,000 times, whereas the SLOW_FUNCTION predicate will be true 1 in 2 times. It clearly made sense to apply QUICK_FUNCTION first to reduce the rowsource as soon as possible. An excerpt from the 10053 trace file in Listing 9-50 shows the workings of the CBO for this query.

Listing 9-50. 10053 Trace File Excerpt for Default Selectivity Statistics

***************************************
BASE STATISTICAL INFORMATION
***********************
Table Stats::
  Table: THOUSAND_ROWS  Alias: THOUSAND_ROWS
    #Rows: 1000  #Blks:  12  AvgRowLen:  55.00  ChainCnt:  0.00
Access path analysis for THOUSAND_ROWS
***************************************
SINGLE TABLE ACCESS PATH
  Single Table Cardinality Estimation for THOUSAND_ROWS[THOUSAND_ROWS]
  No statistics type defined for function SLOW_FUNCTION
  No default cost defined for function SLOW_FUNCTION
  No statistics type defined for function SLOW_FUNCTION
  Default selectivity for function SLOW_FUNCTION: 50.00000000%
  No statistics type defined for function QUICK_FUNCTION
  No default cost defined for function QUICK_FUNCTION
  No statistics type defined for function QUICK_FUNCTION
  Default selectivity for function QUICK_FUNCTION: 0.10000000%
  Table: THOUSAND_ROWS  Alias: THOUSAND_ROWS
    Card: Original: 1000.000000  Rounded: 1  Computed: 0.50  Non Adjusted: 0.50
  Access Path: TableScan
    Cost:  5.10  Resp: 5.10  Degree: 0
      Cost_io: 5.00  Cost_cpu: 3288527
      Resp_io: 5.00  Resp_cpu: 3288527
  Best:: AccessPath: TableScan
         Cost: 5.10  Degree: 1  Resp: 5.10  Card: 0.50  Bytes: 0

The ASSOCIATE STATISTICS command can also be used to supply cost information for PL/SQL functions and this has a similar impact to the selectivity statistics above. Listing 9-51 shows how to set default cost statistics for the GET_RATE PL/SQL function:

Listing 9-51. Setting Default Cost Statistics for a PL/SQL Function

SQL> ASSOCIATE STATISTICS WITH FUNCTIONS get_rate DEFAULT COST (403416, 2, 0);

Statistics associated.

I calculated these cost statistics as follows (in parameter order):

  • CPU cost (403416): Calculated by the DBMS_ODCI.ESTIMATE_CPU_UNITS(<ms>) function, where <ms> is the number of milliseconds it takes to execute the GET_RATE function once (as reported by the Hierarchical Profiler).
  • I/O cost (2): The sum of logical and physical I/Os of one execution of GET_RATE (reported by Autotrace).
  • Network cost (0): This is not yet implemented so it can be left at 0.

It is possible to set both selectivity and costs in a single ASSOCIATE STATISTICS command and it makes sense to supply both when using this strategy.

Extensible Optimizer

I’ll finish this walkthrough of supplying statistics to the CBO with a brief overview of the Extensible Optimizer. This feature (part of the Oracle Data Cartridge toolset), takes default statistics one stage further by associating selectivity and cost by means of an object type, rather than by hardcoded defaults such as those previously mentioned.

It is possible to spend many pages describing the Extensible Optimizer so I’ll just provide a flavor of the technology to show you how to build a method for estimating statistics for your PL/SQL functions that can adapt to your data patterns.

Consider Listing 9-52. I have a simple function that returns a product code in uppercase. Without statistics, the CBO will assume that any predicate using this function will have a 1% selectivity as follows.

Listing 9-52. Default Selectivity for a PL/SQL Function Predicate

SQL> CREATE FUNCTION format_prod_category(
  2                  p_prod_category IN VARCHAR2
  3                  ) RETURN VARCHAR2 DETERMINISTIC IS
  4  BEGIN
  5     RETURN UPPER(p_prod_category);
  6  END format_prod_category;
  7  /

Function created.
SQL> SELECT *
  2  FROM   products
  3  WHERE  format_prod_category(prod_category) = 'SOFTWARE/OTHER';

Execution Plan
----------------------------------------------------------
Plan hash value: 1954719464

-----------------------------------------------
| Id  | Operation         | Name     | Rows  |
-----------------------------------------------
|   0 | SELECT STATEMENT  |          |     1 |
|*  1 |  TABLE ACCESS FULL| PRODUCTS |     1 |
-----------------------------------------------

The CBO has estimated a cardinality of just 1 row for this predicate, but I know for a fact that there are more rows for the Software/Other category as the data profile in Listing 9-53 shows.

Listing 9-53. Data Profile for PRODUCTS Table

SQL> SELECT prod_category
  2  ,      c                                          AS num_rows
  3  ,      ROUND(RATIO_TO_REPORT(c) OVER () * 100, 1) AS selectivity
  4  FROM  (
  5         SELECT prod_category
  6         ,      COUNT(*) AS c
  7         FROM   products
  8         GROUP  BY
  9                prod_category
 10        )
 11  ORDER  BY
 12         num_rows;

PROD_CATEGORY                      NUM_ROWS SELECTIVITY
-------------------------------- ---------- -----------
Hardware                                  2         2.8
Photo                                    10        13.9
Electronics                              13        18.1
Peripherals and Accessories              21        29.2
Software/Other                           26        36.1

You can see a range of selectivity in this data, so a default statistics solution wouldn’t be flexible enough to cover all categories. Therefore, you can use the Extensible Optimizer to build a dynamic statistics parser for this function predicate. As stated earlier, this is implemented using an object type, the specification for which is provided in Listing 9-54.

Listing 9-54. Creating a Statistics Type for the Extensible Optimiser

SQL> CREATE TYPE prod_stats_ot AS OBJECT (
  2  
  3     dummy_attribute NUMBER,
  4
5     STATIC FUNCTION ODCIGetInterfaces (
  6                     p_interfaces OUT SYS.ODCIObjectList
  7                     ) RETURN NUMBER,
  8  
  9     STATIC FUNCTION ODCIStatsSelectivity (
 10                     p_pred_info      IN  SYS.ODCIPredInfo,
 11                     p_selectivity    OUT NUMBER,
 12                     p_args           IN  SYS.ODCIArgDescList,
 13                     p_start          IN  VARCHAR2,
 14                     p_stop           IN  VARCHAR2,
 15                     p_prod_category  IN  VARCHAR2,
 16                     p_env            IN  SYS.ODCIEnv
 17                     ) RETURN NUMBER,
 18  
 19     STATIC FUNCTION ODCIStatsFunctionCost (
 20                     p_func_info      IN  SYS.ODCIFuncInfo,
 21                     p_cost           OUT SYS.ODCICost,
 22                     p_args           IN  SYS.ODCIArgDescList,
 23                     p_prod_category  IN  VARCHAR2,
 24                     p_env            IN  SYS.ODCIEnv
 25                     ) RETURN NUMBER
 26  );
 27  /

Type created.

The Extensible Optimizer uses well-defined interface methods and Listing 9-54 uses the three that are needed to generate selectivity and cost statistics. You must use the exact method names that Oracle prescribes, as I have done.

The parameter data types and order are also prescribed by the Extensible Optimizer, although you can use parameter names of your own choosing, with one notable exception. You must include the same parameter names in your type methods that you have in the PL/SQL function(s) that will eventually be associated with the statistics type. In my case, the FORMAT_PROD_CATEGORY function that I am building this statistics type for has a single parameter named p_prod_category (so I’ve included this in the relevant methods).

The type body implements the statistics type and can include any logic you like that enables the CBO to determine the selectivity and cost of your associated PL/SQL function(s). The type body for the PROD_STATS_OT.ODCIStatsSelectivity method is provided in Listing 9-55 (the remaining methods are available from the Apress web site).

Listing 9-55. Creating a Statistics Type for the Extensible Optimiser (Excerpt of the Type Body)

SQL> CREATE TYPE BODY prod_stats_ot AS
  2  
  3     STATIC FUNCTION ODCIGetInterfaces ...
<snip>
 13     STATIC FUNCTION ODCIStatsSelectivity (
 14                     p_pred_info        IN  SYS.ODCIPredInfo,
 15                     p_selectivity      OUT NUMBER,
 16                     p_args             IN  SYS.ODCIArgDescList,
 17                     p_start            IN  VARCHAR2,
18                     p_stop             IN  VARCHAR2,
 19                     p_prod_category    IN  VARCHAR2,
 20                     p_env              IN  SYS.ODCIEnv
 21                     ) RETURN NUMBER IS
 22     BEGIN
 23  
 24        /* Calculate selectivity of predicate... */
 25        SELECT (COUNT(CASE
 26                         WHEN UPPER(prod_category) = p_start
 27                         THEN 0
 28                      END) / COUNT(*)) * 100 AS selectivity
 29        INTO   p_selectivity
 30        FROM   sh.products;
 31  
 32        RETURN ODCIConst.success;
 33     END ODCIStatsSelectivity;
 34
 35     STATIC FUNCTION ODCIStatsFunctionCost ...
<snip>
 76  
 77  END;
 78  /

As its name suggests, the ODCIStatsSelectivity method is used to calculate the selectivity for the associated PL/SQL function predicate for a given set of values. How it does this is the interesting part. Imagine that I have a predicate of the form WHERE format_prod_category(p.prod_category) = 'SOFTWARE/OTHER'. When the CBO optimizes this predicate, it invokes the ODCIStatsSelectivity method and passes the value ‘SOFTWARE/OTHER’ on to the statistics method in the p_start and p_stop parameters. (If you have a range predicate, p_start and p_stop will contain the lower and upper bounds, respectively.) This means, therefore, that I can specifically count the number of occurrences of ‘SOFTWARE/OTHER’ in the PRODUCTS table to determine the selectivity, as above.

Once the statistics type is created, it can be associated with the PL/SQL function. Listing 9-56 demonstrates the syntax for associating the statistics type and a couple of small queries to show its dynamic nature.

Listing 9-56. Associating and Using the Statistics Type for Dynamic PL/SQL Function Statistics

SQL> ASSOCIATE STATISTICS WITH FUNCTIONS format_prod_category USING prod_stats_ot;

Statistics associated.

SQL> SELECT *
  2  FROM   products
  3  WHERE  format_prod_category(prod_category) = 'SOFTWARE/OTHER';

Execution Plan
----------------------------------------------------------
Plan hash value: 1954719464
-----------------------------------------------
| Id  | Operation         | Name     | Rows  |
-----------------------------------------------
|   0 | SELECT STATEMENT  |          |    26 |
|*  1 |  TABLE ACCESS FULL| PRODUCTS |    26 |
-----------------------------------------------

SQL> SELECT *
  2  FROM   products
  3  WHERE  format_prod_category(prod_category) = 'HARDWARE';

Execution Plan
----------------------------------------------------------
Plan hash value: 1954719464

-----------------------------------------------
| Id  | Operation         | Name     | Rows  |
-----------------------------------------------
|   0 | SELECT STATEMENT  |          |     2 |
|*  1 |  TABLE ACCESS FULL| PRODUCTS |     2 |
-----------------------------------------------

You can see that by using the Extensible Optimizer, I’ve provided accurate and dynamic statistics on my PL/SQL function to the CBO (in this example, these match the data profile of the PRODUCTS table perfectly). Note that the ODCI* methods are called once during the SQL optimization phase (i.e. the “hard parse”) and bind variables are also supported (as long as bind-variable peeking is enabled). However, you should give some consideration to the time it takes to execute your statistics methods and ensure that this is more than offset by the performance improvements they provide or by the fact that you reuse the shared cursor many times over.

images Note If you want to see how the CBO invokes the statistics type methods to extract the selectivity and costs, you can run a 10053 trace and view the resulting trace file.

Tuning PL/SQL

I’ve described many techniques for improving the performance of queries that use PL/SQL functions. Unless you are going to eliminate or dramatically reduce your PL/SQL function calls, however, you should also consider tuning the functions themselves.

You have a range of tuning options at your disposal (especially with later versions of Oracle Database), such as native compilation, subprogram-inlining, new integer data types, array fetching, associative array caching, and so on. For your critical queries and/or heavily-executed PL/SQL functions, you should find that you can reduce their elapsed time by using some of the wide range of PL/SQL tuning techniques available to you.

That said, I’m only going to demonstrate a tuning technique involving array-caching here, because it’s closely related to the Result Cache option I demonstrated earlier and can have dramatic results.

User-Defined PL/SQL Session Cache

I mentioned earlier that caching lookup data in associative arrays is an alternative to the Cross-Session PL/SQL Function Result Cache when the built-in feature is not available. There are several ways to do this and in Listing 9-57, I have provided one such method. I’ve created a PL/SQL array cache for the rates data (using a private global associative array) and provided a single GET_RATES function to load and access the cached data.

Listing 9-57. A User-Defined PL/SQL Cache for Rates

SQL> CREATE OR REPLACE PACKAGE BODY rates_pkg AS
  2  
  3     /* Index subtype for cache... */
  4     SUBTYPE key_st IS VARCHAR2(128);
  5  
  6     /* Rates cache... */
  7     TYPE rates_aat IS TABLE OF rates.exchange_rate%TYPE
  8        INDEX BY key_st;
  9     rates_cache rates_aat;
 10  
 11     /* Cache-enabled function... */
 12     FUNCTION get_rate (
 13              p_rate_date IN rates.rate_date%TYPE,
 14              p_from_ccy  IN rates.base_ccy%TYPE,
 15              p_to_ccy    IN rates.target_ccy%TYPE
 16              ) RETURN rates.exchange_rate%TYPE IS
 17  
 18        v_rate rates.exchange_rate%TYPE;
 19        v_key  key_st := TO_CHAR(p_rate_date, 'YYYYMMDD')
 20                         || '~' || p_from_ccy || '~' || p_to_ccy;
 21  
 22     BEGIN
 23  
 24        IF rates_cache.EXISTS(v_key) THEN
 25  
 26           /* Cache hit... */
 27           v_rate := rates_cache(v_key);
 28  
 29        ELSE
 30  
 31           /* Cache miss. Fetch and cache... */
 32           SELECT exchange_rate INTO v_rate
 33           FROM   rates
 34           WHERE  rate_date  = p_rate_date
 35           AND    base_ccy   = p_from_ccy
 36           AND    target_ccy = p_to_ccy;
 37           rates_cache(v_key) := v_rate;
 38
 39        END IF;
 40
 41        RETURN v_rate;
42
 43     END get_rate;
 44
 45  END rates_pkg;
 46  /

I’ve omitted the package specification (it only has the GET_RATES function signature) and listed just the package body. To keep the example short, I’ve excluded the exception-handling that would be needed to cater for NO_DATA_FOUND, among others. Some other points to note about the implementation are as follows:

Lines 7-9: I’ve created a private global associative array type and variable to store the cached data.

Lines 19-20: The index of the array is a string representation of the primary key for the RATES table. I’ve assigned this key directly in the function’s declaration to save space, but for best practice, any assignments should be performed in the executable block of the PL/SQL program.

Lines 24-27: I test to see if the rate is already cached. If it is, I simply return it directly from the cache.

Lines 32-37: If the rate is not already in the cache, I fetch it from the RATES table and add it to the cache. I’ve decided to cache only what is needed to reduce the potential PGA memory footprint (as an alternative, you might prefer to pre-load small lookup tables into associative arrays on instantiation).

In Listing 9-58, I’ve repeated the sales report that I used in Listing 9-40 (when demonstrating the Result Cache) but replaced the call to the result-cached GET_RATES function with my new user-defined cache equivalent.

Listing 9-58. Query Performance with a User-Defined PL/SQL Session Cache Function

SQL> SELECT t.calendar_year
  2  ,      p.prod_name
  3  ,      SUM(s.amount_sold)                               AS amt_sold_usd
  4  ,      SUM(s.amount_sold *
  5             rates_pkg.get_rate(s.time_id, 'USD', 'GBP')) AS amt_sold_gbp
  6  FROM   sales     s
  7  ,      products  p
  8  ,      times     t
  9  WHERE  s.prod_id = p.prod_id
 10  AND    s.time_id = t.time_id
 11  GROUP  BY
 12         t.calendar_year
 13  ,      p.prod_name
 14  ;

272 rows selected.
Elapsed: 00:00:11.93

Statistics
----------------------------------------------------------
       1591  recursive calls
          0  db block gets
       4729  consistent gets
          0  physical reads
          0  redo size
      16688  bytes sent via SQL*Net to client
        437  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          9  sorts (memory)
          0  sorts (disk)
        272  rows processed

Like the Result Cache example, the user-defined cache has dramatically reduced the logical I/O and runtime of the query to acceptable levels. In fact, the statistics for the user-defined session cache are similar to those of the Result Cache (the cache misses are identical), although the user-cache is slightly slower overall. Listing 9-59 shows the Hierarchical Profiler report for the query and demonstrates where the additional time (over the Result Cache example) was spent.

Listing 9-59. Query Performance with a User-Defined PL/SQL Session-Cache Function (Hierarchical Profiler Report)

FUNCTION                                  LINE#      CALLS SUB_ELA_US FUNC_ELA_US
------------------------------------ ---------- ---------- ---------- -----------
__plsql_vm                                    0     918848    7734487     2202214
RATES_PKG.GET_RATE                           12     918843    5531933     5434184
RATES_PKG.__static_sql_exec_line32           32       1460      97749       97749
__anonymous_block                             0          3        328         270
DBMS_OUTPUT.GET_LINES                       180          2         58          58
RATES_PKG.__pkg_init                          0          1         12          12
DBMS_HPROF.STOP_PROFILING                    59          1          0           0

You can see from the calls to the SQL inside the rates function that there were 1,460 cache misses, which is identical to the earlier scalar subquery cache and Result Cache examples. This accounts for a dramatic reduction in the number of lookups against the RATES table. However, remember that the Result Cache also eliminated the calls to the GET_RATES PL/SQL function itself. In this respect, the user-defined cache is less optimal as calls to the RATES_PKG.GET_RATES function are not reduced at all; in fact, the function calls account for all of the additional runtime.

By using your own PL/SQL array cache, as I have done, you are essentially tuning the function rather than reducing or eliminating its use. Nonetheless, the user-defined PL/SQL session cache is a useful tuning technique for reducing the cost of SQL embedded in PL/SQL functions.

images Caution If you choose your own PL/SQL array cache over the Result Cache, you should be aware of three potential shortcomings. Firstly, each array cache will be visible to a single session and will not be shared. Secondly, each session will need to cache its own copy of the data at the expense of private PGA memory (conversely, the Result Cache stores a single shared copy of its results in the SGA). Thirdly, and most critically, is that if you cache data from tables that are regularly updated, you’ll need some form of cache-management, which will be difficult to implement. The Result Cache comes with this facility “out of the box”, but in your PL/SQL programs, you’ll not have this luxury.

Summary

To wrap up this chapter, I have demonstrated a range of costs that you should consider when designing PL/SQL functions into your applications and queries. In addition, I’ve demonstrated a wide range of techniques that are at your disposal to eliminate, reduce, or optimize the use of PL/SQL functions in your SQL statements.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset