Basic questions about unixCmd and runInTerminal

fmiramar · June 24, 2020, 4:26am

Hello!

I am trying to run external software(CDP, SoX, FFMPEG, etc) using .unixCmd and .runInTerminal and encountering several problems…

When I run the command using .runInTerminal everything works fine, but the problem is that for each command it keeps opening the SCStringForTerminal window… Is it possible to close all of them at once ? Or not open at all ?

When I try to run the command using unixCmd on a mac nothing happens…

// This works
"ffmpeg -i someaudiofile.wav -vn -ar 44100 -ac 2 -b:a 192k someaudiofile.mp3".runInTerminal

// None of this work:
"ffmpeg -i someaudiofile.wav -vn -ar 44100 -ac 2 -b:a 192k someaudiofile.mp3".unixCmd{|res, pid| [\done, res, pid].postln};

"ffmpeg -i someaudiofile.wav -vn -ar 44100 -ac 2 -b:a 192k someaudiofile.mp3".unixCmd

"ffmpeg -i someaudiofile.wav -vn -ar 44100 -ac 2 -b:a 192k someaudiofile.mp3".unixCmdGetStdOut

"ffmpeg -i someaudiofile.wav -vn -ar 44100 -ac 2 -b:a 192k someaudiofile.mp3".systemCmd

What is the problem here ?

Can someone explain in practical terms what is the practical diference between executing a UNIX command asynchronously vs synchronously ? Is this related only to the language or is it related to the server ? It is “synced” with what ?

All the best,
Fellipe Miranda

jamshark70 · June 24, 2020, 4:51am

Synchronous, related to system commands, means that all sclang processing will stop and wait for the command to finish. No other sclang code or threads will be allowed to do anything until the other process exits. Normally this is not what you want.

Asynchronous means that the command runs on its own and sclang keeps doing its thing. The “done” function is the signal telling sclang code that it’s finished (and this is where your code can respond to that).

I can’t guess right now why unixCmd isn’t working for you. I use it rather often and haven’t had this issue. Maybe check system $PATH or environment variables?

hjh

Sam_Pluta · June 24, 2020, 1:14pm

I don’t think the PATH carries over into SC for this kind of thing. I always put the full paths of program and files I am trying to execute in the unixCmd.

So, this doesn’t work for me:

"flac ".unixCmd

but this does:

"/usr/local/Cellar/flac/1.3.3/bin/flac ".unixCmd

and this does:

“/usr/local/Cellar/flac/1.3.3/bin/flac -d /Users/spluta/Downloads/Mahler3B.flac”.unixCmd

Sam

elgiano · June 24, 2020, 1:53pm

I don’t have much experience with macs… even less with brew.
But on linux at least, PATH definitely works in unixCmd:

"echo $PATH".unixCmd // -> outputs my $PATH

jamshark70 · June 24, 2020, 2:53pm

My somewhat loose understanding of it (in Unixy systems) is:

The type of shell that you’re using has an init script, e.g. ~/.bashrc which sets up custom paths. This script is run when opening a terminal window or tab.

But it wouldn’t run when launching the SC interpreter process.

I haven’t tested closely enough to know that this is definitely the case or not, but if true, it would explain why the system can find executables when running a command in a terminal window but not otherwise: if the command’s executable is not in the default system PATH, but the location of the executable is added to the PATH in the init script, then the init script would help find the executable for runInTerminal but not unixCmd.

Consider that an educated guess on my part but I wouldn’t be surprised if that explains the case in the original post.

hjh

MarcinP · June 24, 2020, 5:28pm

I agree that this does seem like a problem with $PATH in SC not containing some of the entries that the system terminal has set. I’ve had a similar problem in the past.

AFAICT typically this will be an issue with executables installed with homebrew since they are linked into /usr/local

You can add a path to $PATH like this:

“PATH”.setenv(“PATH”.getenv ++ “:/usr/local/bin”)

Marcin

fmiramar · June 24, 2020, 7:24pm

Hum… I am still quite lost… Could this be something regarding the fact the Anaconda (Python distribution) is messing PATH or other environment variable ? I am using SC 3.10.4 on Mac Mojave.

My PATH variable is :

/Users/myusername/bin:/anaconda3/bin:/anaconda3/condabin:/Users/myusername/cdpr7/_cdp/_cdprogs:/Users/myusername/.lyp/lilyponds/2.18.2/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin~/usr/local/bin:/usr/local/bin/usr/local/bin

Basically when I run a command on Mac Terminal everything works fine, but when using .unixCmd it doesn’t work…

For instance, when I type ffmpeg on mac terminal it display the current version info, when I execute "ffmpeg ".unixCmd on SC I only get a yellow number on Post Window (something like 1889). Strangely, when I type ffmpeg -help on mac terminal I get the ffmpeg help text and when I execute "ffmpeg -help".unixCmd on SC I now get the expected help on the Post Window…

This inconsistent behavior is also happening with basic comands like:

"ls ".unixCmd on SC displays:

-> 1893
Applications
Library
Network
System
Users
Volumes
anaconda3
bin
cores
dev
etc
home
installer.failurerequests
net
private
sbin
tmp
usr
var

ls on mac terminal displays:

(base) myusername-macbookpro:~ myusername$ ls -1
Applications
Desktop
Documents
Downloads
Google Drive
Library
Movies
Music
OM
OneDrive
Pictures
Public
Zotero
bin
cdp_terminal_sc.py
cdpr7
dwhelper
extras
fm_brassage_pitchmode.wav
fm_incr_freq.wav
fm_modrad_16_0.125.wav
fm_modrad_16_0.25.wav
fm_modrad_16_0.375.wav
fm_modrad_16_0.5.wav
fm_modrad_16_0.625.wav
fm_modrad_16_0.75.wav
fm_modrad_16_0.875.wav
matlab_crash_dump.2124-1
seaborn-data
teste.ipe.autosave
tongues.flac
tongues.wav

fmiramar · June 24, 2020, 7:46pm

For me, all of these are working, even the first one.

fmiramar · June 24, 2020, 7:48pm

Also did not work for me…

MarcinP · June 24, 2020, 8:11pm

Fellipe, have you checked your PATH variable inside SC?

“PATH”.getenv.postln

fmiramar · June 24, 2020, 8:40pm

My PATH on SC, "PATH".getenv.postln :

/Users/myusername/bin:/anaconda3/bin:/anaconda3/condabin:/Users/myusername/cdpr7/_cdp/_cdprogs:/Users/myusername/.lyp/lilyponds/2.18.2/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin~/usr/local/bin:/usr/local/bin/usr/local/bin:/usr/local/bin

My PATH on mac terminal echo $PATH

/Users/myusername/bin:/anaconda3/bin:/anaconda3/condabin:/Users/fellipemirandamartins/cdpr7/_cdp/_cdprogs:/Users/myusername/.lyp/lilyponds/2.18.2/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin

MarcinP · June 24, 2020, 9:11pm

Indeed it seems to that you have most paths in the PATH in SC, however there are some problems after /Library/TeX/texbin - there’s a tilde (~) there, also some paths are not separated with a colon

You could manually rewrite the whole env variable:

"PATH".setenv("/Users/myusername/bin:/anaconda3/bin:/anaconda3/condabin:/Users/myusername/cdpr7/_cdp/_cdprogs:/Users/myusername/.lyp/lilyponds/2.18.2/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/usr/local/bin")

fmiramar · June 24, 2020, 9:55pm

I’ve tried this here and it still have the same issue (diferent ffmpeg and ls behaviors)…

Could it be something regarding the location which SC is being executed ?

"pwd".unixCmd on SC returns:

→ 3401
/

pwd on mac terminal retuns:

/Users/myusername

jamshark70 · June 24, 2020, 10:18pm

If you’re doing something like ""ffmpeg -i someaudiofile.wav ...".unixCmd then yes, the pwd does matter.

In general, never use relative paths to files in unixCmd – always specify a full path, and quote it to help with reserved shell characters.

(Here, I’m talking about someaudiofile.wav.)

hjh

fmiramar · June 25, 2020, 3:33pm

I’ve tried several other things here and none worked properly… I will try install SC 3.11.0 and see if it works.

If it does not work would this be the case for opening a github issue ?

MarcinP · June 25, 2020, 6:06pm

few points:

ls will indeed provide different output between terminal and SC, since SC starts from a different working directory
"ffmpeg".unixCmd will not provide any output (at least it doesn’t on my system) even if ffmpeg is in the PATH - you need to provide parameters to get the output (like ffmpeg -help or the actual encoding command)
what do you mean by “different ffmpeg” in your last post?

In order to see which executable SC is using, you could run
"which ffmpeg".unixCmd
(If that command does not produce output, that means that ffmpeg is not in the PATH in the SC environment.)

Also, have you addressed the issue with the path to the soundfile that James pointed out?

So far this still looks like an issue with paths and not like an actual bug.

RFluff · June 25, 2020, 7:07pm

What you put in bashrc indeed only gets run when bash is interactively invoked. For the rest there’s .profile and/or .bash_profile. But this needs a session restart (e.g. log out and log in again to take effect.) More complete explanation: bash - Difference between .bashrc and .bash_profile - Super User

The op seems to be on a Mac, which has switched to zsh, by the way, if using the latest macOS. I don’t know if they set up zsh to still read the bash init files.

And on older macOS it seems all terminal windows run as login shells, so they read the .profile. And the Mac specific ways of setting the PATH from launchd Setting environment variables via launchd.conf no longer works in OS X Yosemite/El Capitan/macOS Sierra/Mojave? - Stack Overflow

jamshark70 · June 27, 2020, 11:38pm

Looking back at this thread, I think $PATH was a red herring.

So the ffmpeg executable is being found.

Since you’re using relative paths to the media files, and we know that cwd is different in a terminal window and with unixCmd, I’d say the probability is pretty high that the problem is not specifying the full path.

Unix programs can direct text output to stdout (which echoes to the post window in unixCmd) or to stderr. If ffmpeg uses stderr to complain about files that it couldn’t find, you might not see that printed in the post window.

But with:

… when ffmpeg exits, you should definitely have the array posting in SC, so the post window would show

-> number
[ done, exit code, number ]

So I’m still not totally clear on the symptoms.

hjh

fmiramar · June 29, 2020, 6:51am

Indeed, when I enter the fullpath the commands are working, as I pointed when replying to @Sam_Pluta suggestion.

This works fine:

“ffmpeg -i /Users/myusername/someaudiofile.wav -vn -ar 44100 -ac 2 -b:a 192k /Users/myusername/someaudiofile.mp3”.unixCmd

The inconsistent behavior that I find now is that the SC Post Window is not displaying the same text info outputs as the Terminal. Moreover, every time I call one program on Terminal it generally display the current version, the license, the authors and a minimal help, this tends not to happen on SC Post Window.

For instance, comparing the output of this

"flac".unixCmd;
"ffmpeg".unixCmd;
"cython".unixCmd;
"lame".unixCmd;

with the output of this:

"flac".runInTerminal;
"ffmpeg".runInTerminal;
"cython".runInTerminal;
"lame".runInTerminal;

Only flac is displaying the identical text output at both Terminal and SC Postwindow. What could be causing this ? When running "which nameoftheprogram" all of them are located at /anaconda3/bin.

Moreover, when I run

"ffmpeg -i /Users/myusername/someaudiofile.wav -vn -ar 44100 -ac 2 -b:a 192k /Users/myusername/someaudiofile.mp3".unixCmd;
"ffmpeg -i /Users/myusername/someaudiofile.wav -vn -ar 44100 -ac 2 -b:a 192k /Users/myusername/someaudiofile.mp3".runInTerminal;

In the first one I receive no text info at SC Post Window, but in the second one I get something like :

ffmpeg version 4.2 Copyright (c) 2000-2019 the FFmpeg developers
built with clang version 4.0.1 (tags/RELEASE_401/final)
configuration: --prefix=/anaconda3 --cc=x86_64-apple-darwin13.4.0-clang --disable-doc --disable-openssl --enable-avresample --enable-gnutls --enable-gpl --enable-hardcoded-tables --enable-libfreetype --enable-libopenh264 --enable-libx264 --enable-pic --enable-pthreads --enable-shared --enable-static --enable-version3 --enable-zlib --enable-libmp3lame
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Guessed Channel Layout for Input Stream #0.0 : mono
Input #0, wav, from ‘/Users/myusername/someaudiofile.wav’:
Duration: 00:00:20.00, bitrate: 705 kb/s
Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, mono, s16, 705 kb/s
Stream mapping:
Stream #0:0 → #0:0 (pcm_s16le (native) → mp3 (libmp3lame))
Press [q] to stop, [?] for help
Output #0, mp3, to ‘/Users/myusername/someaudiofile.mp3’:
Metadata:
TSSE : Lavf58.29.100
Stream #0:0: Audio: mp3 (libmp3lame), 44100 Hz, stereo, s16p, 192 kb/s
Metadata:
encoder : Lavc58.54.100 libmp3lame
size= 470kB time=00:00:20.01 bitrate= 192.5kbits/s speed=86.6x
video:0kB audio:470kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.139541%
logout
Saving session…
…copying shared history…
…saving history…truncating history files…
…completed.

Could this be related to some verbosity settings ?

(My macOS is using BASH as default)

tedmoore · April 18, 2021, 1:46am

Greetings,

I’m also having this issue. I would like to call a wget with a .unixCmd. .runInTerminal does work, but I’d rather not use it.

Thanks!

T