Sysdig notes for inspecting process and thread activity

I've been looking for a dtrace alternative on linux for a while and recently came accross

Archive note, 8 October 2026. These are 2014 Sysdig notes. The chisel below counts brk events per thread; it does not measure total allocated bytes, live heap size, or all allocation paths. Its descriptions have been corrected to match the counter, without claiming modern compatibility.

sysdig which looks promising.
A few interting one liners:

get all the threads id of a running process (mysqld):

sysdig -p %thread.tid proc.name=mysqld| sort -u

Show the activity of a single thread:

sysdig thread.tid=5445 proc.name=mysqld

The intersting point of sysdig compared to other is the ability to script 'chisel' allowing to write a complex probe, the default install comes with a few

Here's one of mine, getting the top allocating threads of a process:

--[[
Author: Jérémie Banier
Contact: jbanier@gmail.com
Date: 08 Sep 2014
 
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License version 2 as
published by the Free Software Foundation.


This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see .
--]]

-- The number of items to show
TOP_NUMBER = 30

-- Chisel description
description = "Show the top " .. TOP_NUMBER .. " threads ranked by brk event count. You can use filters to restrict this to a specific process, thread or file."
short_description = "Top threads by brk event count"
category = "Performance"

-- Chisel argument list
args = {}

-- Argument notification callback
function on_set_arg(name, val)
 return false
end

-- Initialization callback
function on_init()
 chisel.exec("table_generator", 
  "thread.tid",
  "Thread ID",
  "evt.count",
  "# Calls",
  "evt.type=brk",
  "" .. TOP_NUMBER,
  "none")
 return true
end

(save it in ~/.chisels then invoke it with sysdig -c topmemory proc.name=mysqld)

Sadly it still miss a few dtrace feature like enabling a probe when you got into specific user land function or the ability to display a stack trace but the tool is still young :)

There's a JSON output format available which means you could trace all reads and writes, send them to kibana and graph them or trace all syscall failing with ENOMEM, EAGAIN, ... graph and know when you're getting behind in terms of capacity. all kind of cool stuff :)