When a C Program Morphs Into Machine Language

Advertisements

Memory Layout of A Running Program (Process)

When a program is scheduled to run by the CPU, it allocates memory space for it. Specific part of this allocated memory is used for a specific purpose. Memory layout of a process pictorially represents different segments of the process’s memory to help us visualize what-why-how of the program logic as it runs, the understanding of which enables us to debug the program effectively.

There are six distinct segments in the memory layout of a process, each of which serves a different purpose as depicted in the following diagram.
memory-layout-of-a-process-csea

  1. Text segment: This is the actual code to be executed by the CPU. This area of memory is sharable and multiple instances of the program make use of a common copy of text segment to lower memory requirements. And is usually read-only so the program can’t edit its own code once loaded into the memory.
  2. Initialized data segment: This contains the “global variables” initialized by the programmer.
  3. Uninitialized data segment (bss): As the name suggests this segment contains all the uninitialized global variables, and are initialized to 0 (zero) or NULL pointer before the program begins to execute. This segment is also known as bss (Block Started by Symbol), an old assembly operator used by few old assemblers.
  4. Stack: Stack is a special memory area used by processes to keep track of the flow of execution during function calls. It’s a collection of stack frames, each of which corresponds to a function call.
  5. Heap: Heap is also a special memory area used by processes when they need memory “on the fly”. It’s the most dynamic memory in that chunks of memory allocated, coalesced and merged frequently to effectively manage the free space. When a programmer uses malloc() and friends or new, he is explicitly marking the usage of heap at run time.
  6. Commandline arguments & environment variables: The higher part of memory stores the commandline arguments and other environment variables if required by the program.

Given an object file or executable, you can see the size of each segment. Do note that these are files on the disk which eventually become residents of memory. Consider the following program.

// Memory layout learning
#include<stdio.h>

char banner[] = "Hello World";
int main()
{
   printf("%s\n",banner);
   return 0;
}

To compare the size of different sections, compile and link separately.

$ gcc -c hello.c
$ gcc -o hello hello.o

The size command can be used to list the various sections in the object file (hello.o) and the executable (hello).

$ size hello.o hello
   text	   data	    bss	    dec	    hex	filename
     77	     12	      0	     89	     59	hello.o
   1170	    564	      4	   1738	    6ca	hello

Here data is the combined size of initialized and uninitialized data segments. The dec and hex values represent the total size in decimal and hexadecimal respectively.

The size of the sections of object file can also be obtained by running objdump -h or objdump -x.

$ objdump -h hello.o

hello.o:     file format elf64-x86-64

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         00000015  0000000000000000  0000000000000000  00000040  2**0
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
  1 .data         0000000c  0000000000000000  0000000000000000  00000058  2**3
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000000  0000000000000000  0000000000000000  00000064  2**0
                  ALLOC
  3 .comment      00000035  0000000000000000  0000000000000000  00000064  2**0
                  CONTENTS, READONLY
  4 .note.GNU-stack 00000000  0000000000000000  0000000000000000  00000099  2**0
                  CONTENTS, READONLY
  5 .eh_frame     00000038  0000000000000000  0000000000000000  000000a0  2**3
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA

Think?!
size and objdump report different sizes for the text segment. Can you guess where the discrepancy comes from? Hint: How big is the discrepancy? See anything of that length in the source code?

Compile and Run C Program From Within VIM

For those of us who write C programs in Vim, nothing could be more frustrating than to leave the editor (:x, :wq, shift+ZZ) to compile and run the source code we are working on. Here are a few productivity-enhancing tips in this regard.

1. “!”
The bang “!” symbol allows the execution of arbitrary commands inside the Vim editor. To compile and run the current source code file run,

:! gcc % && ./a.out

Here % is substituted by the currently opened file. Vim displays the output in a blank shell screen until we press Enter. Forget not to save the source file (:w) before you compile since Vim runs the command on the file, not on the buffer content.

vim-bang-command-gcc

In order to compile a specific file,

:!gcc somecode.c -o somecode && ./somecode

2. “make”
The Vim :make command can be used to build the source code from the Makefile. The simplest Makefile would contain,

program
        gcc hello.c

We can write a Makefile that encompasses our entire project.

3. “keymap”
The best solution is to associate an hotkey to compile-and-run just like in an IDE. Open .vimrc (create one if you don’t have) and add,

map <F8> :w <CR> :!gcc % -o %< && ./%< <CR>

Here “:w” saves the current file, and “%<” is the file name without the extension. Now every time you hit F8, Vim compiles the source code, creates the object file (without any file extension) and runs it for us.

C Tricky Question: Print “Hello World” Without Using Any Semicolon In The Program

Unlike a natural language such as English, a formal language such as C adheres to strict code of conduct in which no redundancy, ambiguity or word-play exists. Every statement in C must terminate with a semicolon for it to be syntactically correct. However, the conditional statements such as “if” contains two parts:

  1. Header — where the truth-test of the logical expression is evaluated
  2. Body — statement(s) to be executed should the logical expression evaluates to true (non-zero)

Considering if itself as a micro-function, only its body need to be terminated with semi-colon. Neither its header nor its closing brace should have a semi-colon. This is the key to answer the aforementioned question. Just embed the print statement inside the if’s logical condition and leave its body statement-less.

/*Program to print "Hello World" or any message 
without using any “;” in the program*/

#include <stdio.h>
void main()
{
    if(printf("Hello World\n"))
    { }
}

c-no-semicolon-hello-world-output

The C Program Journey

Ever wondered what happens behind the scenes when you compile a C program? If you have, you’re at the right place since this post demystifies everything that happens when you compile the C program. As it turns out, the journey of a C program from the human readable source file to the final executable has four stages.

Before we delve deep into each of those stages, just for the sake of context, let’s quickly go through the typical process of compiling a C program. We are using gcc and the GNU toolchain, the de facto compiler and build system for C in Linux.

Here’s our sample C program named hello.c :

#include <stdio.h>

#define STRING "hello world\n"

int main(void)
{

	// printing by substituting the macro
	printf(STRING);

	return 0;
}

Next we compile the above source code using gcc like so:

$ gcc hello.c -o hello

Compiling a C program

Note here that hello.c is the source file to be compiled by gcc. The -o (oh) option tells gcc what should be the name of the executable file so compiled (hello). If -o option is omitted, by default gcc names the executable file as “a.out“.

Lastly, we execute the compiled file hello to see the result which in this case just prints “hello world” on the screen.

Executing helloworld program C

What just happened was a transformation from the source code to the executable through four specific stages. They are summarized in the following diagram.

Journey of C program

By default, gcc takes care of all the four stages one after the other to produce the executable. We can instruct gcc to do only what we want by specifying the right command-line switch. Let’s examine each of the four stages in detail.

Stage #1 : Pre-processing

The first stage is pre-processing during which following actions take place:

  1. Macro substitution
  2. Comments are stripped off
  3. Header files are expanded

The pre-processor accepts the .c file and unless specified by the -o switch, the output is echoed onto the stdout.

gcc pre-processing

gcc -E hello_c

Let’s examine the hello.i file.

# 1 "hello.c"
# 1 ""
# 1 ""
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "" 2
# 1 "hello.c"
# 1 "/usr/include/stdio.h" 1 3 4
...
...
...
  # 873 "/usr/include/stdio.h" 3 4
extern FILE *popen (const char *__command, const char *__modes) ;

extern int pclose (FILE *__stream);

extern char *ctermid (char *__s) __attribute__ ((__nothrow__ , __leaf__));

# 913 "/usr/include/stdio.h" 3 4
extern void flockfile (FILE *__stream) __attribute__ ((__nothrow__ , __leaf__));
extern int ftrylockfile (FILE *__stream) __attribute__ ((__nothrow__ , __leaf__)) ;
extern void funlockfile (FILE *__stream) __attribute__ ((__nothrow__ , __leaf__));

# 943 "/usr/include/stdio.h" 3 4

# 2 "hello.c" 2

int main(void)
{

 printf("hello world");

 return 0;
}

Three things can be noticed from the large pre-processed file.

First, the macro STRING has been substituted with its character string value. Second, the comment we wrote above the printf statement has been stripped off. Third, the header file stdio.h has been expanded with hundreds of lines of code. By this we get to know that the header file source code actually gets inserted into our source file.

If we search for printf, we’ll get the following:

extern int printf (const char *__restrict __format, ...);

The keyword ‘extern’ tells that the function printf() is not defined here. It is external to this file. We will later see how gcc gets the definition of printf().

Stage #2 : Compilation

The second stage is compilation in which the GNU C compiler accepts the pre-processed hello.i file and outputs the compiled file named hello.s. Note here that the compiler expects its input file’s extension to be “.i.

gcc compilation

gcc -S hello_i

Viewing hello.s file reveals that the C tokens and instructions have been replaced with assembly language directives and instructions.

		.file	"hello.c"
	.section	.rodata
.LC0:
	.string	"hello world"
	.text
	.globl	main
	.type	main, @function
main:
.LFB0:
	.cfi_startproc
	pushq	%rbp
	.cfi_def_cfa_offset 16
	.cfi_offset 6, -16
	movq	%rsp, %rbp
	.cfi_def_cfa_register 6
	movl	$.LC0, %edi
	movl	$0, %eax
	call	printf
	movl	$0, %eax
	popq	%rbp
	.cfi_def_cfa 7, 8
	ret
	.cfi_endproc
.LFE0:
	.size	main, .-main
	.ident	"GCC: (Debian 4.9.2-10) 4.9.2"
	.section	.note.GNU-stack,"",@progbits

Stage #3 : Assembly

The third stage is assembly in which the compiled output is passed to the assembler. The assembler expects its input file’s extension to be “.s” and produces an intermediate file with the extension “.o“.

gcc assembly

gcc -c hello_s

The compiled file hello.s in the previous stage is nothing but a bunch of assembler directives to be interpreted by the assembler. GCC internally calls the GNU Assembler as to do the job of interpreting the assembly level instructions in the compiled file to produce the machine level code. This machine code is also known as the object code. You can also call the as independently to process hello.s instead via gcc like so:

$ as hello.s -o hello.o

At this stage only the existing code is converted into machine language, the function calls such as printf() are not resolved.

Since the output of this stage is a machine level file (hello.o), its content is not understandable by us. If we still try to open the hello.o and view it, we’ll see something that is totally not readable.

ELF object file

The only thing we can explain by looking at the print.o file is about the string ELF. ELF stands for executable and linkable format. This is a relatively new format for machine level object files and executables that are produced by gcc. Prior to this, a format known as a.out was used. ELF is said to be a format that’s more sophisticated than a.out.

Note that if you compile your code without specifying the name of the output file, the output file produced has name ‘a.out’, but the format now have changed to ELF. The default executable file name has nothing to do with the format of the machine code. The same name a.out is mere incidental.

Stage #4 : Linking

This is the last stage in which some housekeeping functions are performed by the linker to produce the ready-to-run machine level code. Calling gcc without any option will link all the object files to produce the final executable.

gcc linking

gcc hello.o

As discussed earlier, till this stage gcc doesn’t know about the definition of functions like printf(). Until the compiler knows exactly where all of these functions are implemented, it simply uses a place-holder for the function call. It is at this stage, the definition of printf() is resolved and the actual address of the function printf() is plugged in.

gcc internally makes use of the GNU Linker ld to achieve this task. You can directly call ld to link the object files like so:

$ ld hello.o -o hello

The linker also does some extra work; it adds extra code to our program that is required to indicate when the program starts and when the program ends. For example, there is code which is standard for setting up the running environment like passing command line arguments, passing environment variables to every program. Similarly some standard code that is required to return the return value of the program to the system.

The above tasks of the compiler can be verified by a small experiment. Since now we already know that the linker converts .o file (hello.o) to an executable file (hello). If we compare the file sizes of both the hello.o and hello file, we’ll see the difference.

size hello.o and hello

Through the size command we get a rough idea about how the size of the output file increases from an object file to an executable file. This is all because of that extra standard code the linker adds to our program.

That’s all there to it about what happens when you compile a C program. Isn’t this beautiful!

Pro Tip: You can pass the –save-temp switch to gcc to get all the intermediate files in one command.

$ gcc --save-temp hello.c -o hello

gcc --save-temp