| Updated: October 28, 2024 |
Control the scheduler
#include <sys/sched_aps.h>
#include <sys/neutrino.h>
int SchedCtl( int cmd,
void *data,
int length);
int SchedCtl_r( int cmd,
void *data,
int length);
If you've loaded the adaptive partitioning module into your OS image, the following commands are also available:
For details about each command and its data, see the sections below.
libc
Use the -l c option to qcc to link against this library. This library is usually included automatically.
The SchedCtl() and SchedCtl_r() kernel calls control the scheduler. These functions are identical except in the way they indicate errors; see the Returns section for details.
The adaptive partitioning scheduler is optional and is present only if you add [module=aps] to your OS image's buildfile. For more information, see the Adaptive Partitioning User's Guide.
For the control commands that are related to adaptive partitioning, you must initialize all of the fields—including reserved ones—in the structures you pass as the data argument, by calling (for example) memset(). You can also use the APS_INIT_DATA() macro:
APS_INIT_DATA( &data );
SCHED_APS_QUERY_PARMS
This command fills in a sched_aps_info structure that describes the overall parameters of the adaptive partitioning scheduler:
typedef struct {
uint64_t cycles_per_ms;
uint64_t windowsize_cycles; /* Deprecated */
uint64_t windowsize2_cycles; /* Deprecated */
uint64_t windowsize3_cycles; /* Deprecated */
uint32_t scheduling_policy_flags;
uint32_t sec_flags;
uint32_t bankruptcy_policy;
uint16_t num_partitions;
uint16_t max_partitions;
uint16_t windowsize_ms;
uint16_t reserved1;
uint32_t reserved2;
uint64_t reserved3;
} sched_aps_info;
The members include:
Scheduling policies
These flags set options for the adaptive partitioning scheduling algorithm. To set, pass a pointer to an ORed set of these flags with the SCHED_APS_SET_PARMS call to SchedCtl():
By default, the scheduler hands out free time to the partition with the highest-priority running thread. That guarantees realtime scheduling behavior (i.e., scheduling strictly by priority) to partitions any time they aren't being limited by some other partition's right to its guaranteed minimum budget. But it also means that one partition is allowed to grab all the free time.
If you set SCHED_APS_SCHEDPOL_FREETIME_BY_RATIO, the running partitions share the free time in proportion to the ratios of their budgets. So, one partition can no longer grab all the free time. However, when this flag is set, partitions will see strict priority-scheduling between partitions only when they're consuming less than their CPU budgets.
Allowing the highest-priority partition to run to completion means lower-priority partitions won't run in the meantime. That means that when the system is loaded, the default algorithm can cause small-budget low-priority partitions to see long delays between intervals when they run. (For example, on a loaded system with a 100 ms averaging window, a 10% partition may run only every 90 milliseconds.) One way to reduce the latency is to reduce the averaging window size.
The SCHED_APS_SCHEDPOL_PARTITION_LOCAL_PRIORITIES policy schedules purely by budget ratio. When enabled, the scheduler tries to balance budgets on as short a timescale as possible, regardless of the window size.
That means high-priority partitions with large budgets no longer run to completion (while they have budget). They're timesliced with other partitions, even low-priority ones. The result is that SCHED_APS_SCHEDPOL_PARTITION_LOCAL_PRIORITIES can reduce the latencies seen by small budget partitions in a loaded system. This comes at the cost of a departure from strict priority preemptive behavior when all partitions have budget (i.e., the default policy).
The SCHED_APS_SCHEDPOL_PARTITION_LOCAL_PRIORITIES policy provides the latencies while retaining the full accuracy of a 100 ms averaging window. It continues to schedule the highest priority thread within partitions.
This option includes the behavior of SCHED_APS_SCHEDPOL_FREETIME_BY_RATIO.
When a critical thread consumes critical time, it temporarily forces a return to the default scheduling policy. Critical threads are allowed to run to completion and aren't timesliced by SCHED_APS_SCHEDPOL_PARTITION_LOCAL_PRIORITIES's attempts to balance budgets. In other words, critical threads aren't affected by this policy.
Scheduling within a partition is always strictly by priority, no matter which of these flags are set.
For more information about adaptive partitioning and BMP, see the Adaptive Partitioning Scheduling Details chapter of the Adaptive Partitioning User's Guide.
Handling bankruptcy
Bankruptcy is when critical CPU time billed to a partition exceeds its critical budget. Bankruptcy is always considered to be a design error on the part of the application, but you can configure how the system responds to it.
If the system isn't declaring bankruptcy when you expect it, note that bankruptcy can be declared only if critical time is billed to your partition. Critical time is billed on those timeslices when the following conditions are all met:
Only then if the critical time, billed over the current averaging window, exceeds a partition's critical budget will the system declare the partition bankrupt.
When the system detects that a partition has gone bankrupt:
In addition, you can configure the following responses:
To set a choice of bankruptcy-handling options, OR the above SCHED_APS_BNKR_* flags and pass a pointer to it as the bankruptcy_policyp field of the sched_aps_parms structure when you call SCHED_APS_SET_PARMS.
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_QUERY_PARMS command (see the Returns section for details):
SCHED_APS_SET_PARMS
This command sets the parameters for the overall behavior of the adaptive partitioning scheduler. The data argument must be a pointer to a sched_aps_parms structure:
typedef struct {
int16_t windowsize_ms;
int16_t reserved1;
uint32_t *scheduling_policy_flagsp;
uint32_t *bankruptcy_policyp;
int32_t reserved2;
int64_t reserved3;
} sched_aps_parms;
The members include:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_SET_PARMS command (see the Returns section for details):
For more information, see “Security,” below.
SCHED_APS_CREATE_PARTITION
This command creates a new partition that's considered to be a child of the partition that's calling SchedCtl(). The system automatically creates a partition called System (the value of APS_SYSTEM_PARTITION_NAME) with an ID of 0.
The data argument for this command must be a pointer to a sched_aps_create_parms structure:
typedef struct {
/* input parms */
char *name;
uint16_t budget_percent;
int16_t critical_budget_ms;
uint8_t aps_create_flags;
int8_t parent_id;
uint16_t max_budget_percent;
uint16_t critical_priority;
uint16_t budget_percent_scale;
uint64_t reserved1;
uint32_t reserved2;
/* output parms */
int16_t id;
int16_t reserved3;
} sched_aps_create_parms;
The input members include:
Actual Budget (%) = budget_percent / (10 ^ budget_percent_scale)
The output members include:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_CREATE_PARTITION command (see the Returns section for details):
For more information, see “Security,” below.
SCHED_APS_QUERY_PARTITION
This command gets information about a given partition. The data argument for this command must be a pointer to a sched_aps_partition_info structure:
typedef struct {
/* out parms */
uint64_t budget_cycles; /* Deprecated */
uint64_t critical_budget_cycles;
char name[APS_PARTITION_NAME_LENGTH+1];
int16_t parent_id;
uint16_t budget_percent;
int32_t notify_pid; /* Deprecated */
int32_t notify_tid; /* Deprecated */
uint32_t pinfo_flags; /* Deprecated */
int32_t pid_at_last_bankruptcy;
int32_t tid_at_last_bankruptcy;
uint16_t max_budget_percent;
uint16_t critical_priority;
uint16_t budget_percent_scale;
int16_t reserved1;
int64_t reserved2;
/* input parm */
int16_t id;
} sched_aps_partition_info;
The input members include:
The output members include:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_QUERY_PARTITION command (see the Returns section for details):
SCHED_APS_LOOKUP
This command finds the partition ID for a given partition name.
The data argument for this command must be a sched_aps_lookup_parms structure:
typedef struct {
/* input parms */
char *name;
int16_t reserved1;
/* output parms */
int16_t id;
} sched_aps_lookup_parms;
The input members include:
The output members include:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_LOOKUP command (see the Returns section for details):
SCHED_APS_JOIN_PARTITION
This command makes the thread specified by the given process and thread IDs becomes a member of the specified partition. This partition also becomes the thread's new home partition, i.e., where it returns after partition inheritance.
The data argument for this command must be a pointer to a sched_aps_join_parms structure:
typedef struct {
int16_t id;
int16_t reserved1;
int32_t pid;
int32_t tid;
int32_t aid;
} sched_aps_join_parms;
The members include:
If tid is -2, then the partition in which pulses are handled is changed, plus all threads in the given process are joined to the target partition.
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_JOIN_PARTITION command (see the Returns section for details):
For more information, see “Security,” below.
SCHED_APS_MODIFY_PARTITION
This command changes the parameters of an existing partition. If the new budget's value is different from the current, the difference is either taken from, or returned to, the parent partition's budget. The critical time parameter affects only the chosen partition, not its parent. To change just one of new budget or new critical time, set the other to -1.
The data argument for this command must be a pointer to a sched_aps_modify_parms structure:
typedef struct {
int16_t id;
int16_t new_budget_percent;
int16_t new_critical_budget_ms;
int16_t new_max_budget_percent;
int16_t new_critical_priority;
uint16_t budget_percent_scale;
int32_t reserved1;
int64_t reserved2;
} sched_aps_modify_parms;
The members include:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_MODIFY_PARTITION command (see the Returns section for details):
For more information, see “Security,” below.
SCHED_APS_PARTITION_STATS
This command returns instantaneous values of the CPU time-accounting variables for a set of partitions. It can fill in data for more than one partition. If the length argument to SchedCtl() indicates that you've passed the function an array of sched_aps_partition_stats structures, SchedCtl() fills each element with statistics for a different partition, starting with the partition specified by the id field.
To determine the number of partitions, use the SCHED_APS_OVERALL_STATS command.
The command overwrites the id field with the partition number for which data is being returned. It stores -1 in the id field of unused elements.
To convert times in cycles into milliseconds, divide them by the cycles_per_ms obtained with an SCHED_APS_QUERY_PARMS command.
The data argument for this command must be a pointer to a sched_aps_partition_stats structure, or an array of these structures:
typedef struct {
/* out parms */
uint64_t run_time_cycles;
uint64_t critical_time_cycles;
uint64_t run_time_cycles_w2; /* Deprecated */
uint64_t critical_time_cycles_w2; /* Deprecated */
uint64_t run_time_cycles_w3; /* Deprecated */
uint64_t critical_time_cycles_w3; /* Deprecated */
uint32_t stats_flags;
uint32_t reserved1;
uint64_t dynamic_windowsize_cycles; /* length of last averaging window used for scheduling */
uint64_t reserved2;
/* in parm */
int16_t id;
} sched_aps_partition_stats;
The members include:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_PARTITION_STATS command (see the Returns section for details):
SCHED_APS_OVERALL_STATS
This command returns instantaneous information about scheduler states. The data argument for this command must be a pointer to a sched_aps_overall_stats structure:
typedef struct {
uint64_t idle_cycles; /* Deprecated */
uint64_t idle_cycles_w2; /* Deprecated */
uint64_t idle_cycles_w3; /* Deprecated */
int16_t id_at_last_bankruptcy;
uint16_t reserved1;
int32_t pid_at_last_bankruptcy;
int32_t tid_at_last_bankruptcy;
uint32_t reserved2;
uint32_t reserved3;
uint64_t reserved4;
} sched_aps_overall_stats;
The members include:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_OVERALL_STATS command (see the Returns section for details):
SCHED_APS_QUERY_THREAD
This command determines the partition for the given thread and indicates whether or not the thread in your process is marked to run as critical. Use a thread ID of zero to indicate the calling thread.
The data argument for this command must be a pointer to a sched_aps_query_thread_parms structure:
typedef struct {
int32_t pid;
int32_t tid;
/* out parms: */
int16_t id;
int16_t inherited_id;
uint32_t crit_state_flags;
int32_t reserved1;
int32_t reserved2;
} sched_aps_query_thread_parms;
The input members include:
The output members include:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_QUERY_THREAD command (see the Returns section for details):
SCHED_APS_ADD_SECURITY
This command sets security options. A bit that's set turns the corresponding security option on. Successive calls add to the existing set of security options. Security options can be cleared only by a restart.
The data argument for this command must be a pointer to a sched_aps_security_parms structure:
typedef struct {
uint32_t sec_flags;
uint32_t reserved1;
uint32_t reserved2;
} sched_aps_security_parms;
The members include:
Security
The adaptive partitioning scheduler lets you dynamically create and modify the partitions in your system.
However you might need to modify a partition at runtime. In this case, you can use the security options described below.
When QNX Neutrino starts, it sets the security option to SCHED_APS_SEC_OFF. We recommend that you immediately set it to SCHED_APS_SEC_RECOMMENDED. In code, do this:
sched_aps_security_parms p; APS_INIT_DATA( &p ); p.sec_flags = SCHED_APS_SEC_RECOMMENDED; SchedCtl( SCHED_APS_ADD_SECURITY,&p, sizeof(p) );
The security options include the following:
Unless you're testing the partitioning and want to change all parameters without needing to restart, you should set at least SCHED_APS_SEC_BASIC.
In general, SCHED_APS_SEC_RECOMMENDED is more secure than SCHED_APS_SEC_FLEXIBLE, which is more secure than SCHED_APS_SEC_BASIC. All three allow partitions to be created and modified. After setting up partitions, use SCHED_APS_SEC_PARTITIONS_LOCKED to prevent further unauthorized changes. For example:
sched_aps_security_parms p; APS_INIT_DATA( &p ); p.sec_flags = SCHED_APS_SEC_PARTITIONS_LOCKED; SchedCtl( SCHED_APS_ADD_SECURITY, &p, sizeof(p));
SCHED_APS_SEC_RECOMMENDED, SCHED_APS_SEC_FLEXIBLE, and SCHED_APS_SEC_BASIC are composed of the flags defined below (but it's usually more convenient for you to use the compound options):
#define SCHED_APS_SEC_BASIC (SCHED_APS_SEC_ROOT0_OVERALL | SCHED_APS_SEC_ROOT_MAKES_CRITICAL)
#define SCHED_APS_SEC_FLEXIBLE (SCHED_APS_SEC_BASIC | SCHED_APS_SEC_NONZERO_BUDGETS |\
SCHED_APS_SEC_ROOT_MAKES_PARTITIONS |\
SCHED_APS_SEC_PARENT_JOINS | SCHED_APS_SEC_PARENT_MODIFIES )
#define SCHED_APS_SEC_RECOMMENDED (SCHED_APS_SEC_FLEXIBLE | SCHED_APS_SEC_SYS_MAKES_PARTITIONS |\
SCHED_APS_SEC_SYS_JOINS | SCHED_APS_SEC_JOIN_SELF_ONLY)
#define SCHED_APS_SEC_OFF 0x00000000
The individual flags are as follows:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_ADD_SECURITY command (see the Returns section for details):
SCHED_APS_QUERY_PROCESS
This command returns the partition of the given process. The partition of a process is billed while one of the process's threads handles a pulse. The individual threads in a process may all be in different partitions from the process.
The data argument for this command must be a pointer to a sched_aps_query_process_parms structure:
typedef struct {
int32_t pid;
/* out parms: */
int16_t id; /* partition of process */
int16_t reserved1;
int64_t reserved2;
int64_t reserved3;
int32_t reserved4;
} sched_aps_query_process_parms;
The members include:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_APS_QUERY_PROCESS command (see the Returns section for details):
SCHED_CONFIGURE (QNX Neutrino 7.0.1 or later)
As a trade-off between latency and throughput, the scheduling algorithm makes decisions that people who expect strict priority scheduling occasionally find surprising:
Immediately scheduling the higher-priority thread to run on CPU A results in lower thread latency for the high-priority thread, but at the expense of decreased total throughput, as it incurs the cost of interrupting the lower-priority thread and possibly moving it to another CPU.
The end result is that only the highest-priority thread in the system is guaranteed to run on some CPU, while the other CPUs may be running threads with lower priorities than those ready to run at any given point in time. The SCHED_CONFIGURE command provides some parameters that you can use to tune the scheduler.
The data argument for this command must be a pointer to a struct sched_config. This structure includes the following members:
Keeping both parameters at their default values (INT_MAX) maintains the scheduling behavior described above.
For example, suppose there are three threads, T20, T10, and T5, where the priority of each thread matches the thread's label, and that T10 is running on CPU A, and T5 is running on CPU B. If thread T20 becomes ready on CPU A, then the results are as follows:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_CONFIGURE command (see the Returns section for details):
SCHED_CONT_APP and SCHED_STOP_APP (QNX Neutrino 7.0 or later)
The SCHED_STOP_APP and SCHED_CONT_APP commands make all processes with the given application ID stop or continue. “Stop” means that no thread in any of these processes is scheduled, and “continue” means allowing these threads to be scheduled again, subject to their current state. You use these commands like this (for more information, see "Application groups" in the QNX Neutrino System Security Guide):
if ( SchedCtl( SCHED_STOP_APP, (void *)appid, 0) == -1 ) {
/* An error occurred */
}
...
if ( SchedCtl( SCHED_CONT_APP, (void *)appid, 0) == -1 ) {
/* An error occurred */
}
This mechanism is independent of the SIGSTOP and SIGCONT signals; for example, a SIGCONT doesn't resume a thread that belongs to a process that was stopped by a SchedCtl(SCHED_STOP_APP, ...) call. Nevertheless, in order to successfully issue these commands, your process must have the PROCMGR_AID_SIGNAL ability enabled such that you could send the appropriate signal:
For more information about PROCMGR_AID_SIGNAL, see procmgr_ability().
This mechanism interacts with the _NTO_TCTL_ONE_THREAD_CONT and _NTO_TCTL_ONE_THREAD_HOLD commands for ThreadCtl() as follows:
Errors:
SchedCtl() and SchedCtl_r() indicate the following errors for the SCHED_CONT_APP and SCHED_STOP_APP commands (see the Returns section for details):
Blocking states
These calls don't block.
The only difference between these functions is the way they indicate errors:
sched_aps_partition_info part_info;
// You need to initialize the parameter block.
APS_INIT_DATA(&part_info);
// Set the input members of the parameter block.
part_info.id = 2;
// Invoke SchedCtl to query the partition.
ret = SchedCtl( SCHED_APS_QUERY_PARTITION, &part_info,
sizeof(part_info) );
if (EOK!=ret) some_kind_of_error_handler();
// Use output field
printf( "The budget is %d per cent.\n",
part_info.budget_percent);
| Safety: | |
|---|---|
| Cancellation point | No |
| Interrupt handler | No |
| Signal handler | Yes |
| Thread | Yes |