1 Introduction to Coverage
Typically in verification flow, we do code coverage, but doing code coverage shows only if all the lines of the DUT is executed, if all the possible cases of a expression are covered.
Which is kind of ok to start with, but it is no way good for saying that verification is completed.
For declaring a verification to be complete we need to check also if all the functions of DUT are tested or not. Like to say if read followed by write to same address of a memory did it happen or to say if 64 Bytes packets happened when port mirroring is enabled.
Typically when we start documenting test cases in test plan document, we also start documenting functional coverage plan. So when we start coding testbench, we code coverage along testbench. E language gives very good set of features that are useful in measuring coverage.
2. Content:
3. Defining Coverage Groups
Defines a coverage group. A coverage group is struct member that contains a list of data items for which data is collected over time. Once data coverage items have been defined in a coverage group, they can be used to define special coverage group items called transition and cross items. The is keyword can also be used to define a new coverage group.
The sampling event of a coverage group cannot be defined in a when subtype of a struct. However, a coverage group that uses an event defined in a parent struct can be defined in a when subtype. If they are extended by adding a per_instance item, the instances refer only to the when subtype. If a per_instance item is instead defined in a base type and additional items are added under the when construct, then the cover group instances refer to the base type and the cover item values refer to the when subtype.
The empty keyword can be used to define an empty coverage group that will be extended later by using a cover is also struct member with the same name
Syntax
cover event-type [using coverage-group-option, ...] is {
coverage-item-definition; ...
};
Example Define Coverage Group
1 <'
2 type cpu_opcode: [ADD, SUB, OR, AND, JMP, LABEL];
3 type cpu_reg: [reg0, reg1, reg2, reg3];
4
5 struct inst {
6 opcode: cpu_opcode;
7 op1: cpu_reg;
8 op2: byte;
9 event info;
10 event data_change;
11 event inst_driven;
12 cover data_change using no_collect is {
13 item data: uint(bits:16) = op2;
14 };
15 cover info is {
16 item opcode;
17 };
18 cover inst_driven is {
19 item opcode;
20 item op1;
21 };
22
23 post_generate() is also {
24 emit info;
25 emit data_change;
26 emit inst_driven;
27 };
28 };
29
30 extend sys {
31 inst : inst;
32 run() is also {
33 for {var i: uint = 0; i < 10 ; i = i + 1} do {
34 gen inst;
35 print inst;
36 };
37 };
38 };
39 '>
Simulation log
inst = inst-@0: inst of unit: sys
---------------------------------------------- @coverage1
0 opcode: ADD
1 op1: reg3
2 op2: 59
inst = inst-@1: inst of unit: sys
---------------------------------------------- @coverage1
0 opcode: OR
1 op1: reg3
2 op2: 90
inst = inst-@2: inst of unit: sys
---------------------------------------------- @coverage1
0 opcode: SUB
1 op1: reg3
2 op2: 188
inst = inst-@3: inst of unit: sys
---------------------------------------------- @coverage1
0 opcode: JMP
1 op1: reg3
2 op2: 97
inst = inst-@4: inst of unit: sys
---------------------------------------------- @coverage1
0 opcode: OR
1 op1: reg1
2 op2: 105
inst = inst-@5: inst of unit: sys
---------------------------------------------- @coverage1
0 opcode: LABEL
1 op1: reg2
2 op2: 4
inst = inst-@6: inst of unit: sys
---------------------------------------------- @coverage1
0 opcode: SUB
1 op1: reg1
2 op2: 14
inst = inst-@7: inst of unit: sys
---------------------------------------------- @coverage1
0 opcode: LABEL
1 op1: reg3
2 op2: 204
inst = inst-@8: inst of unit: sys
---------------------------------------------- @coverage1
0 opcode: JMP
1 op1: reg2
2 op2: 170
inst = inst-@9: inst of unit: sys
---------------------------------------------- @coverage1
0 opcode: LABEL
1 op1: reg0
2 op2: 250
Wrote 1 cover_struct to coverage1_1.ecov
Coverage report Cover group: inst.info
======================
Grade: 0.83 Weight: 1
** opcode **
Samples: 11 Tests: 1 Grade: 0.83 Weight: 1
grade goal samples tests %t opcode
------------------------------------------------
1.00 1 1 1 9 ADD
1.00 1 2 1 18 SUB
1.00 1 3 1 27 OR
0.00 1 0 0 0 AND
1.00 1 2 1 18 JMP
1.00 1 3 1 27 LABEL
Cover group: inst.inst_driven
=============================
Grade: 0.92 Weight: 1
** opcode **
Samples: 11 Tests: 1 Grade: 0.83 Weight: 1
grade goal samples tests %t opcode
------------------------------------------------
1.00 1 1 1 9 ADD
1.00 1 2 1 18 SUB
1.00 1 3 1 27 OR
0.00 1 0 0 0 AND
1.00 1 2 1 18 JMP
1.00 1 3 1 27 LABEL
** op1 **
Samples: 11 Tests: 1 Grade: 1.00 Weight: 1
grade goal samples tests %t op1
------------------------------------------------
1.00 1 1 1 9 reg0
1.00 1 2 1 18 reg1
1.00 1 2 1 18 reg2
1.00 1 6 1 55 reg3
Example Turn off Coverage
1 <'
2 extend sys {
3 setup() is also {
4 set_config(cover, mode, off); //turns off coverage collection
5 };
6 };
7 '>
Defining Basic Coverage Items
Coverage item can be defined with below syntax
item item-name[:type=exp] [using coverage-item-option, ...]
Parameter
Description
item-name
The name of the coverage item.
type=exp
Used to declare a new item.
coverage-item-option
Coverage item options as listed in below table
Options
Option | Description |
at_least=num | The minimum number of samples for each bucket of the item. Anything less than num is considered a hole. |
ignore=item-bool-exp | Define values that are to be completely ignored. They do not appear in the statistics at all. The expression is a Boolean expression that can contain a coverage item name and constants. |
illegal=item-bool-exp | Define values that are illegal. An illegal value causes a DUT error. If the check_illegal_immediately coverage configuration option is FALSE, the DUT error occurs during the check_test phase of the test. If that configuration option is TRUE, the DUT error occurs immediately (on the fly). Note that checking illegal values immediately has a significant negative impact on Specman performance. |
no_collect=bool | When no_collect[=TRUE], this coverage item is not displayed in coverage reports and is not saved in the coverage files. |
no_trace=bool | When no_trace[=TRUE], this item is not traced by the simulator. Use this option to collect data for echo event |
num_of_buckets=num | Specifies how many buckets are to be created for the coverage item. This option overrides the max_int_buckets coverage configuration option for this coverage item. It cannot be used for string items or items that have ranges specified with the ranges coverage item option. It can be set to any value, including greater than max_int_buckets |
per_instance=bool | When per_instance[=TRUE], coverage data is collected and graded for all the other items in a separate sub-group for each bucket of this item. This option can only be used for gradeable items. |
radix=DEC|HEX|BIN | For items of type int or uint, specifies the radix used in coverage reports for implicit buckets. If the ranges option is not used to create explicit buckets for an item, a bucket is created for every value of the item that occurs in the test. Each different value sampled gets its own bucket, with the value as the name of the bucket. These are called implicit buckets. |
ranges = {range(parameters);...} | Create buckets for this item's values or ranges of values. This option cannot be used for string items. |
text=string | A text description for this coverage item. This can only be a quoted string, not a variable or expression. In the ASCII coverage report the text is shown along with the item name at the top of the coverage information for the item. In the Show Coverage GUI, the text is shown for each item. |
weight=uint | Specifies the weight of the current item relative to other items in the same coverage group. It is a non-negative integer with a default of 1. |
when=bool-exp | The item is sampled only when bool-exp is TRUE. The bool-exp is evaluated in the context of the parent struct. |
Example
1 <'
2 type cpu_opcode: [ADD, SUB, OR, AND, JMP, LABEL];
3 type cpu_reg: [reg0, reg1, reg2, reg3];
4
5 struct inst {
6 opcode: cpu_opcode;
7 op1: cpu_reg;
8 op2: byte;
9 event info;
10 event data_change;
11 event inst_driven;
12 cover data_change using no_collect is {
13 item data: uint(bits:16) = op2;
14 };
15 cover info is {
16 item opcode using per_instance, ignore = (opcode == ADD);
17 };
18 cover inst_driven is {
19 item opcode;
20 item op1 using at_least=2;
21 };
22
23 post_generate() is also {
24 emit info;
25 emit data_change;
26 emit inst_driven;
27 };
28 };
29
30 extend sys {
31 inst : inst;
32 run() is also {
33 for {var i: uint = 0; i < 10 ; i = i + 1} do {
34 gen inst;
35 print inst;
36 };
37 };
38 };
39 '>
Simulation Output
inst = inst-@0: inst of unit: sys
---------------------------------------------- @coverage3
0 opcode: ADD
1 op1: reg3
2 op2: 59
inst = inst-@1: inst of unit: sys
---------------------------------------------- @coverage3
0 opcode: OR
1 op1: reg3
2 op2: 90
inst = inst-@2: inst of unit: sys
---------------------------------------------- @coverage3
0 opcode: SUB
1 op1: reg3
2 op2: 188
inst = inst-@3: inst of unit: sys
---------------------------------------------- @coverage3
0 opcode: JMP
1 op1: reg3
2 op2: 97
inst = inst-@4: inst of unit: sys
---------------------------------------------- @coverage3
0 opcode: OR
1 op1: reg1
2 op2: 105
inst = inst-@5: inst of unit: sys
---------------------------------------------- @coverage3
0 opcode: LABEL
1 op1: reg2
2 op2: 4
inst = inst-@6: inst of unit: sys
---------------------------------------------- @coverage3
0 opcode: SUB
1 op1: reg1
2 op2: 14
inst = inst-@7: inst of unit: sys
---------------------------------------------- @coverage3
0 opcode: LABEL
1 op1: reg3
2 op2: 204
inst = inst-@8: inst of unit: sys
---------------------------------------------- @coverage3
0 opcode: JMP
1 op1: reg2
2 op2: 170
inst = inst-@9: inst of unit: sys
---------------------------------------------- @coverage3
0 opcode: LABEL
1 op1: reg0
2 op2: 250
Wrote 1 cover_struct to coverage3_1.ecov
Coverage
Cover group: inst.info
======================
Grade: 0.80 Weight: 1
** opcode **
Samples: 10 Tests: 1 Grade: 0.80 Weight: 1
grade goal samples tests %t opcode
------------------------------------------------
1.00 1 2 1 20 SUB
1.00 1 3 1 30 OR
0.00 1 0 0 0 AND
1.00 1 2 1 20 JMP
1.00 1 3 1 30 LABEL
Cover group: inst.inst_driven
============================
Grade: 0.85 Weight: 1
** opcode **
Samples: 11 Tests: 1 Grade: 0.83 Weight: 1
grade goal samples tests %t opcode
------------------------------------------------
1.00 1 1 1 9 ADD
1.00 1 2 1 18 SUB
1.00 1 3 1 27 OR
0.00 1 0 0 0 AND
1.00 1 2 1 18 JMP
1.00 1 3 1 27 LABEL
** op1 **
Samples: 11 Tests: 1 Grade: 0.88 Weight: 1
grade goal samples tests %t op1
------------------------------------------------
0.50 2 1 1 9 reg0
1.00 2 2 1 18 reg1
1.00 2 2 1 18 reg2
1.00 2 6 1 55 reg3
Defining Cross Coverage Items
Cross items are combinations of items from the same coverage group. The cross coverage construct is used to define cross items.
Syntax
cross item-name-1, item-name-2, ... [using coverage-item-option, ...]
Where | ||
item-name-1, item-name-2, ... | Each item name must be one of the following. | . |
. | The name of an item defined previously in the current coverage group | . |
. | The name of a transition item defined previously in the current coverage group | . |
. | The name of a cross item defined previously in the current coverage group | . |
coverage-item- option | An option for the cross item. | . |
| ||||||||||||||||||
Example |
1 <'
2 type cpu_opcode: [ADD, SUB, OR, AND, JMP, LABEL];
3 type cpu_reg: [reg0, reg1, reg2, reg3];
4
5 struct inst {
6 opcode: cpu_opcode;
7 op1: cpu_reg;
8 op2: byte;
9 event info;
10 event data_change;
11 event inst_driven;
12 cover data_change using no_collect is {
13 item data: uint(bits:16) = op2;
14 };
15 cover info is {
16 item opcode using per_instance, ignore = (opcode == ADD);
17 };
18 cover inst_driven is {
19 item opcode;
20 item op1 using at_least=2;
21 cross opcode, op1;
22 };
23
24 post_generate() is also {
25 emit info;
26 emit data_change;
27 emit inst_driven;
28 };
29 };
30
31 extend sys {
32 inst : inst;
33 run() is also {
34 for {var i: uint = 0; i < 10 ; i = i + 1} do {
35 gen inst;
36 print inst;
37 };
38 };
39 };
40 '>
Simulation Output
inst = inst-@0: inst of unit: sys
---------------------------------------------- @coverage4
0 opcode: ADD
1 op1: reg3
2 op2: 59
inst = inst-@1: inst of unit: sys
---------------------------------------------- @coverage4
0 opcode: OR
1 op1: reg3
2 op2: 90
inst = inst-@2: inst of unit: sys
---------------------------------------------- @coverage4
0 opcode: SUB
1 op1: reg3
2 op2: 188
inst = inst-@3: inst of unit: sys
---------------------------------------------- @coverage4
0 opcode: JMP
1 op1: reg3
2 op2: 97
inst = inst-@4: inst of unit: sys
---------------------------------------------- @coverage4
0 opcode: OR
1 op1: reg1
2 op2: 105
inst = inst-@5: inst of unit: sys
---------------------------------------------- @coverage4
0 opcode: LABEL
1 op1: reg2
2 op2: 4
inst = inst-@6: inst of unit: sys
---------------------------------------------- @coverage4
0 opcode: SUB
1 op1: reg1
2 op2: 14
inst = inst-@7: inst of unit: sys
---------------------------------------------- @coverage4
0 opcode: LABEL
1 op1: reg3
2 op2: 204
inst = inst-@8: inst of unit: sys
---------------------------------------------- @coverage4
0 opcode: JMP
1 op1: reg2
2 op2: 170
inst = inst-@9: inst of unit: sys
---------------------------------------------- @coverage4
0 opcode: LABEL
1 op1: reg0
2 op2: 250
Wrote 1 cover_struct to coverage4_1.ecov
Coverage
Cover group: inst.info
======================
Grade: 0.80 Weight: 1
** opcode **
Samples: 10 Tests: 1 Grade: 0.80 Weight: 1
grade goal samples tests %t opcode
------------------------------------------------
1.00 1 2 1 20 SUB
1.00 1 3 1 30 OR
0.00 1 0 0 0 AND
1.00 1 2 1 20 JMP
1.00 1 3 1 30 LABEL
Cover group: inst.inst_driven
=============================
Grade: 0.71 Weight: 1
** opcode **
Samples: 11 Tests: 1 Grade: 0.83 Weight: 1
grade goal samples tests %t opcode
------------------------------------------------
1.00 1 1 1 9 ADD
1.00 1 2 1 18 SUB
1.00 1 3 1 27 OR
0.00 1 0 0 0 AND
1.00 1 2 1 18 JMP
1.00 1 3 1 27 LABEL
** op1 **
Samples: 11 Tests: 1 Grade: 0.88 Weight: 1
grade goal samples tests %t op1
------------------------------------------------
0.50 2 1 1 9 reg0
1.00 2 2 1 18 reg1
1.00 2 2 1 18 reg2
1.00 2 6 1 55 reg3
** cross__opcode__op1 **
Samples: 11 Tests: 1 Grade: 0.42 Weight: 1
grade goal samples tests %p/%t opcode/op1
------------------------------------------------------
0.25 - 1 1 9/9 ADD
0.00 1 0 0 0/0 ADD/reg0
0.00 1 0 0 0/0 ADD/reg1
0.00 1 0 0 0/0 ADD/reg2
1.00 1 1 1 100/9 ADD/reg3
0.50 - 2 1 18/18 SUB
0.00 1 0 0 0/0 SUB/reg0
1.00 1 1 1 50/9 SUB/reg1
0.00 1 0 0 0/0 SUB/reg2
1.00 1 1 1 50/9 SUB/reg3
0.50 - 3 1 27/27 OR
0.00 1 0 0 0/0 OR/reg0
1.00 1 1 1 33/9 OR/reg1
0.00 1 0 0 0/0 OR/reg2
1.00 1 2 1 67/18 OR/reg3
0.00 - 0 0 0/0 AND
0.50 - 2 1 18/18 JMP
0.00 1 0 0 0/0 JMP/reg0
0.00 1 0 0 0/0 JMP/reg1
1.00 1 1 1 50/9 JMP/reg2
1.00 1 1 1 50/9 JMP/reg3
0.75 - 3 1 27/27 LABEL
1.00 1 1 1 33/9 LABEL/reg0
0.00 1 0 0 0/0 LABEL/reg1
1.00 1 1 1 33/9 LABEL/reg2
1.00 1 1 1 33/9 LABEL/reg3
Defining Transition Coverage Items
Transition items are items for which value changes are collected in the coverage data. The transition coverage group item is used to define transition items.
Syntax
transition item-name [using coverage-item-option, ...]
Where
Parameter | Description | . | |||
item-name | A coverage item defined previously in the current coverage group. | . | |||
coverage-item- option | An option for the cross item. | . | |||
- | Options | ||||
- | Option | Description | . | ||
- | at_least=num | The minimum number of samples for each bucket of the item. Anything less than num is considered a hole. | . | ||
- | ignore=item-bool-exp | Define values that are to be completely ignored. They do not appear in the statistics at all. The expression is a Boolean expression that can contain a coverage item name and constants. | . | ||
- | illegal=item-bool-exp | Define values that are illegal. An illegal value causes a DUT error. If the check_illegal_immediately coverage configuration option is FALSE, the DUT error occurs during the check_test phase of the test. If that configuration option is TRUE, the DUT error occurs immediately (on the fly). Note that checking illegal values immediately has a significant negative impact on Specman performance. | . | ||
- | name=label | Specifies a name for a cross coverage item. No white spaces are allowed in the label. The default is cross__item-a__item-b | . | ||
- | no_collect=bool | When no_collect[=TRUE], this coverage item is not displayed in coverage reports and is not saved in the coverage files. | . | ||
- | per_instance=bool | When per_instance[=TRUE], coverage data is collected and graded for all the other items in a separate sub-group for each bucket of this item. This option can only be used for gradeable items. | - | ||
- | text=string | A text description for this coverage item. This can only be a quoted string, not a variable or expression. In the ASCII coverage report the text is shown along with the item name at the top of the coverage information for the item. In the Show Coverage GUI, the text is shown for each item. | - | ||
- | weight=uint | Specifies the weight of the current item relative to other items in the same coverage group. It is a non-negative integer with a default of 1. | - | ||
- | when=bool-exp | The item is sampled only when bool-exp is TRUE. The bool-exp is evaluated in the context of the parent struct. | - | ||
-- | - | - | - | -- | - |
Example
1 <'
2 type cpu_state: [START, FETCH1, FETCH2, EXEC];
3 struct cpu {
4 st: cpu_state;
5 event state_change;
6 cover state_change is {
7 item st;
8 transition st;
9 };
10
11 post_generate() is also {
12 emit state_change;
13 }
14 };
15
16 extend sys {
17 cpu : cpu;
18 run() is also {
19 for {var i: uint = 0; i < 10 ; i = i + 1} do {
20 gen cpu;
21 print cpu;
22 };
23 };
24 };
25 '>
Simulation Output
cpu = cpu-@0: cpu of unit: sys
---------------------------------------------- @coverage5
0 st: START
cpu = cpu-@1: cpu of unit: sys
---------------------------------------------- @coverage5
0 st: START
cpu = cpu-@2: cpu of unit: sys
---------------------------------------------- @coverage5
0 st: EXEC
cpu = cpu-@3: cpu of unit: sys
---------------------------------------------- @coverage5
0 st: START
cpu = cpu-@4: cpu of unit: sys
---------------------------------------------- @coverage5
0 st: FETCH1
cpu = cpu-@5: cpu of unit: sys
---------------------------------------------- @coverage5
0 st: FETCH2
cpu = cpu-@6: cpu of unit: sys
---------------------------------------------- @coverage5
0 st: EXEC
cpu = cpu-@7: cpu of unit: sys
---------------------------------------------- @coverage5
0 st: START
cpu = cpu-@8: cpu of unit: sys
---------------------------------------------- @coverage5
0 st: START
cpu = cpu-@9: cpu of unit: sys
---------------------------------------------- @coverage5
0 st: START
Wrote 1 cover_struct to coverage5_1.ecov
Coverage
Cover group: cpu.state_change
=============================
Grade: 0.69 Weight: 1
** st **
Samples: 11 Tests: 1 Grade: 1.00 Weight: 1
grade goal samples tests %t st
------------------------------------------------
1.00 1 6 1 55 START
1.00 1 1 1 9 FETCH1
1.00 1 1 1 9 FETCH2
1.00 1 3 1 27 EXEC
** transition__st **
Samples: 10 Tests: 1 Grade: 0.38 Weight: 1
grade goal samples tests %t prev_st=>st
------------------------------------------------
1.00 1 3 1 30 START=>START
1.00 1 1 1 10 START=>FETCH1
0.00 1 0 0 0 START=>FETCH2
1.00 1 1 1 10 START=>EXEC
0.00 1 0 0 0 FETCH1=>START
0.00 1 0 0 0 FETCH1=>FETCH1
1.00 1 1 1 10 FETCH1=>FETCH2
0.00 1 0 0 0 FETCH1=>EXEC
0.00 1 0 0 0 FETCH2=>START
0.00 1 0 0 0 FETCH2=>FETCH1
0.00 1 0 0 0 FETCH2=>FETCH2
1.00 1 1 1 10 FETCH2=>EXEC
1.00 1 3 1 30 EXEC=>START
0.00 1 0 0 0 EXEC=>FETCH1
0.00 1 0 0 0 EXEC=>FETCH2
0.00 1 0 0 0 EXEC=>EXEC
| --------------------------------------------------------------------------- |
Bạn Có Đam Mê Với Vi Mạch hay Nhúng - Bạn Muốn Trau Dồi Thêm Kĩ Năng
Mong Muốn Có Thêm Cơ Hội Trong Công Việc
Và Trở Thành Một Người Có Giá Trị Hơn
Mong Muốn Có Thêm Cơ Hội Trong Công Việc
Và Trở Thành Một Người Có Giá Trị Hơn