I'm trying to understand how stack works in Linux. I read AMD64 ABI sections about stack and process initialization and it is not clear how the stack should be mapped. Here is the relevant quote (3.4.1):
Stack State
This section describes the machine state that
exec(BA_OS) creates for new processes.
and
It is unspecified whether the data and stack segments are initially mapped with execute permissions or not. Applications which need to execute code on the stack or data …
In the x86-64 System V ABI it is specified that the space behind the $rsp - 128 is the so-called red zone which is not touched by any signal handlers. On my machine
$ ulimit -s
8192
I expected there is only 2 pages in the stack. So I wrote the following program to test till which size red zone can expand:
PAGE_SIZE equ 0x1000
SYS_exit equ 0x3C
section .text
global _start
_start:
lea rcx, [rsp - 0x1f * PAGE_SIZE] …