| .TH PCILMR 8 "@TODAY@" "@VERSION@" "The PCI Utilities" |
| .SH NAME |
| pcilmr \- margin PCIe Links |
| .SH SYNOPSIS |
| .B pcilmr |
| .RB [ "--margin" ] |
| .RI [ "<margining options>" ] " <downstream component> ..." |
| .br |
| .B pcilmr --full |
| .RI [ "<margining options>" ] |
| .br |
| .B pcilmr --scan |
| .SH CONFIGURATION |
| List of the requirements for links and system settings |
| to run the margining test. |
| |
| .B BIOS settings |
| (depends on the system, relevant for server baseboards |
| with Xeon CPUs): |
| .IP \[bu] 3 |
| Turn off PCIe Leaky Bucket Feature, Re-Equalization and Link Degradation; |
| .IP \[bu] |
| Set Error Thresholds to 0; |
| .IP \[bu] |
| Intel VMD for NVMe SSDs - in case of strange behavior of the |
| .BR pcilmr, |
| try to run it with the VMD turned off. |
| .PP |
| .B Device (link) requirements: |
| .IP |
| .I "Configured by the user before running the utility, the utility does not change them:" |
| .RS |
| .IP \[bu] 3 |
| The current Link data rate must be 16.0 GT/s or higher (right now |
| utility supports 16 GT/s and 32 GT/s Links); |
| .IP \[bu] |
| Link Downstream Component must be at D0 Power Management State. |
| .RE |
| .IP |
| .I "Configured by the utility during operation, utility set them to their original " |
| .I "state after receiving the results:" |
| .RS |
| .IP \[bu] 3 |
| The ASPM must be disabled in both the Downstream Port and Upstream Port; |
| .IP \[bu] |
| The Hardware Autonomous Speed Disable bit of the Link Control 2 register must be Set in both the |
| Downstream Port and Upstream Port; |
| .IP \[bu] |
| The Hardware Autonomous Width Disable bit of the Link Control register must be Set in both the |
| Downstream Port and Upstream Port. |
| .SH DESCRIPTION |
| .B pcilmr |
| utility allows you to take advantage of the PCIe Lane Margining at the Receiver |
| capability which is mandatory for all Ports supporting a data rate of 16.0 GT/s or |
| higher, including Pseudo Ports (Retimers). Lane Margining at Receiver enables system |
| software to obtain the margin information of a given Receiver while the Link is in the |
| L0 state. The margin information includes both voltage and time, in either direction from |
| the current Receiver position. Margining support for timing is required, while support |
| for voltage is optional at 16.0 GT/s and required at 32.0 GT/s and higher data rates. Also, |
| independent time margining and independent voltage margining is optional. |
| |
| Utility allows to get an approximation of the eye margin diagram in the form of a rhombus |
| (by four points). Lane Margining at the Receiver capability enables users to margin PCIe |
| links without a hardware debugger and without the need to stop the target system. Utility |
| can be useful to debug link issues due to receiver margins. |
| |
| However, the utility results may be not particularly accurate and, as it was found out during |
| testing, specific devices provide rather dubious capability support and the reliability of |
| the information they provide is questionable. The PCIe specification provides reference values |
| for the eye diagram, which are also used by the |
| .B pcilmr |
| to evaluate the results, but it seems that it makes sense to contact the |
| manufacturer of a particular device for references. |
| |
| The PCIe Base Specification Revision 5.0 sets allowed range for Timing Margin from 20%\~UI to 50%\~UI and |
| for Voltage Margin from 50\~mV to 500\~mV. Utility uses 30%\~UI as the recommended |
| value for Timing - taken from NVIDIA presentation ("PCIe 4.0 Mass Electrical Margins Data |
| Collection"). |
| |
| .B pcilmr |
| requires root privileges (to access Extended Configuration Space), but during our testing |
| there were no problems with the devices and they successfully returned to their normal initial |
| state after the end of testing. |
| |
| .SH OPTIONS |
| .SS Device Specifier |
| .B "<device/component>" \t |
| .RI [ "<domain>" :] <bus> : <dev> . <func> |
| (see |
| .BR lspci (8)) |
| .SS Utility Modes |
| .TP |
| .BI --margin " <downstream component> ..." |
| Margin selected Links. |
| .TP |
| .B --full |
| Margin all ready for testing (in a meaning similar to the |
| .B --scan |
| option) Links in the system (one by one). |
| .TP |
| .B --scan |
| Scan for Links with negotiated speed 16 GT/s or higher. Mark "Ready" those of them |
| in which at least one of the Link sides have Margining Ready bit set meaning that |
| these Links are ready for testing and you can run utility on them. |
| .SS Margining Test options |
| .TP |
| .B -c |
| Print Device Lane Margining Capabilities only. Do not run margining. |
| .TP |
| \fB\-l\fI <lane>\fP[\fI,<lane>...\fP] |
| Specify lanes for margining. |
| .br |
| Remember that Device may use Lane Reversal for Lane numbering. However, utility |
| uses logical lane numbers in arguments and for logging. Utility will automatically |
| determine Lane Reversal and tune its calls. |
| .br |
| Default: all link lanes. |
| .TP |
| .BI -e " <errors>" |
| Specify Error Count Limit for margining. |
| .br |
| Default: 4. |
| .TP |
| \fB-r\fI <recvn>\fP[\fI,<recvn>...\fP] |
| Specify Receivers to select margining targets. |
| .br |
| Default: all available Receivers (including Retimers). |
| .TP |
| .BI -p " <parallel_lanes>" |
| Specify number of lanes to margin simultaneously. |
| .br |
| According to spec it's possible for Receiver to margin up to MaxLanes + 1 |
| lanes simultaneously, but during testing, performing margining on several |
| lanes simultaneously led to results that were different from sequential |
| margining, so this feature requires additional verification and |
| .I -p |
| option right now is for experiments mostly. |
| .br |
| Default: 1. |
| .PP |
| .B "Use only one of -T/-t options at the same time (same for -V/-v)." |
| .br |
| .B "Without these options utility will use MaxSteps from Device" |
| .B "capabilities as test limit." |
| .TP |
| .B -T |
| Time Margining will continue until the Error Count is no more |
| than an Error Count Limit. Use this option to find Link limit. |
| .TP |
| .BI -t " <steps>" |
| Specify maximum number of steps for Time Margining. |
| .TP |
| .B -V |
| Same as |
| .I -T |
| option, but for Voltage. |
| .TP |
| .BI -v " <steps>" |
| Specify maximum number of steps for Voltage Margining. |
| .SS Margining Log options |
| .TP |
| .BI -o " <directory>" |
| Save margining results in csv form into the specified directory. Utility |
| will generate file with the name in form of |
| .RI "\[dq]lmr_" "<downstream component>" "_Rx" # _ <timestamp> ".csv\[dq]" |
| for each successfully tested receiver. |
| |
| .SH EXAMPLES |
| Utility syntax example: |
| .RS |
| .BI "pcilmr -l" " 0,1 " "-r" " 1,6 " "-TV" " ab:0.0 52:0.0" |
| .RE |
| |
| .UR https://gist.github.com/bombanya/f2b15263712757ffba1a11eea011c419 |
| Examples of collected results on different systems. |
| .UE |
| |
| .SH SEE ALSO |
| .nh |
| .BR lspci (8), |
| .B PCI Express Base Specification (Lane Margining at Receiver) |
| .hy |