[Geany-Devel] Placeholder replacement in (build) commands

Fri Apr 3 00:52:33 UTC 2015

[...]
>> Are you sure they don't?  see
>>
>> http://blogs.msdn.com/b/twistylittlepassagesallalike/archive/2011/04/23/everyone-quotes-arguments-the-wrong-way.aspx
>>
>> Says they work, no better than they work on sh though :)
>
>
> To find proper quoting for Windows, you'll need to look no further than
> spawn_append_argument() in the new spawn. :)

Well, thats obviously the name of a quoting function, perhaps it
should be spawn_append_argument_win_quoted() so I might have noticed
it :)

Its fine for appending an argument, but when replacing a placeholder
you can't be certain it isn't inside quotes already.  Which is the
original b4n point.  So we don't yet have a working placeholder
replacement quoter (which is I think what you say below where you are
thinking more :).

But it doesn't put arguments containing tabs inside " either.

>
> The only thing I'm not sure about are \n \v ... - the official VS 2013 msdn
> [1]

[1] was what I was looking for when I found the blog, thanks :)

> says "arguments are delimited by white space, which is either a space or
> a tab", but the correct(?) solution in this blog includes \n and \v without
> any explanation. And it may slightly different under the various C RTL-s for
> Win32.

It puts arguments containing \n and \v inside " which sounds
reasonable but yeah unexplained why.

>
> I was aware of this problem, and that's why, even in the oldest spawn
> versions, the blank characters in a Windows command line are #define-d, and
> may be easily changed.
>
>>> I should have clarified: if the original command, before any placeholder
>>> expansion, does not contain any quotes. If there is at least a single
>>> quote,
>>> we assume the user knows what (s)he is doing.
>>
>>
>> Well, its a simple rule, but on win, paths ending in backslash will
>> break because you get prog "c:\dir with spaces\" and the closing quote
>> is escaped.  So its no improvement on the current situation.
>
>
> What I did not consider is that our placeholders may end in directory
> separator (pretty obvious actually, \ is the root Win~1 directory).
> In that case, spawn_append_argument() will do the job.

Yep.

>
>> Note that the quotes on the C filetype build commands may be because
>> of the system() hack, since it passes stuff through cmd before
>> creating the process and cmd has *another* set of quotes, see the link
>> above.
>
>
> I know, and have no intention to support the cmd rules, neither in spawn nor
> anywhere else.

Agreed, we are trying to get rid of it :)

>
>> Also if I read the win rules right, quoting %d would prevent you doing
>> "%d\filename with spaces" or "%d\%f" to get absolute paths, since it
>> doesn't do the catenation of separately quoted parts that sh does (as
>> I read it).  On sh you would do %d'\filename with spaces' or %d\\%f.

Ok, your reference at [1] says it does do catenation, so to answer
myself %d"\filename with spaces" should work on win if %d is quoted,
same as sh.

[...]
> I know... :) We have working natively quoting and escaping and breaking and
> whatever else is required for native quotation functions for Unix
> (g_shell_quote) and Windows (spawn_append_argument), so maybe we can stop
> discussing these details?

As b4n originally said, and I think you agree below, except if the
command we are interpolating into has quotes in it already.

[...]
> So, in my understanding, you want to drop the shell syntax, and that is why
> you want to drop /bin/sh. But what stops "our own thing" from producing
> command line suitable for g_shell_parse_argv, and thus /bin/sh?

I wasn't actually serious about dropping sh, how would we annoy b4n
with pipes and subshells :)

>
>>> '$amounts'. Our current double-quoted placeholders make it "$amounts", so
>>> the users users of Geany can have fun.
>>
>>
>> As the shell substitutes whatever amounts is defined to be :) But
>> maybe I *want* to substitute $CFLAGS into the command.
>
>
> File and directory placeholders.

Sorry I wasn't clear, these refer to the non-build placeholders that
are not specifically files or directories.  Where the selection is
used for example.  But hopefully any quotes that are added will get
removed by g_shell_parse_argv() when it splits the command so the
$substitutions will work again.

But g_shell_quote() only works if the text being quoted is a complete
argument, not if its being interpolated inside existing quotes, eg if
a user used `/bin/sh -c "%s"` as a context action expecting the
selection to be executed as a shell command.  Note the documented
example for context actions includes quotes around the placeholder.

>
> Now, if you *want* a file or directory placeholder to result in $CFLAGS, and
> then $CFLAGS to be expanded as a shell variable, the current double-quoted
> Geany build commands should be perfect for you, because they do exactly
> that. :)

Yeah, I doubt you would want to substitute *into* a filename.

[...]
> Remember that
> we must rewrite the placeholders replacement to *at least* do all
> replacements at once, and maybe add %% for literal % (though it may be
> currently escaped as \% under Unix and quoted with "%" under Windows).

I thought that the %% was just so a command could contain the literal
sequence %f, %d etc, so its just something in
build_replace_placeholders() not a quoting issue?

>
>> For me right means:
>>
>> - that it "just works" most of the time,
>> - there are no situations where the user is screwed totally and can't
>> get what they want, and
>> - existing quoting either still works or crashes loudly, it never
>> silently does something different.
>
>
> A reasonable minimum.
>
> Personally I wanted to:
>
> - considerably improve the current situation
> - with something simple (and it's simple, see "lots" above)

Some of the ideas proposed by b4n don't meet my definition of "simple"
in that its not obvious they are right and that they don't screw
somebody :)

> - without breaking anything, even if that means not fixing something
>
> But my idea was no good.
>
> First, the default Geany commands are improperly double quoted, so we can
> expect their user-modified derivatives to retain the double quotes, which
> makes the idea less useful, because of the "no auto quoting if the original
> command line contains at least one quote of any kind".
> And it probably does. Gee.
>
> Second, if we unquote the placeholders in all default commands, they will be
> auto quoted, but only until the user adds a probably unrelated quote, and
> then they'll break, which is unacceptable.
>
> So I need some more time to think.

No problem, take your time.

>
>> And there is still those commands that are not run in the shell to
>> consider, how are they quoted so they split up right.
>
>
> Once again, they are run shell-like, see above.

See above (and b4n's original post) for the problem.

>
>> And windows.
>
>
> And bash under Windows (part of MSYS), if we want to be complete.

Sigh, making win folks install even more weird foreign programs is
probably not the best solution, OTOH it could allow build commands to
always be run under sh even on win so only one solution to the quoting
problem is needed.

Cheers
Lex

>
>> Sigh!
>
>
> -''-
>
> [1] https://msdn.microsoft.com/en-us/library/17w5ykft.aspx
>
>
> --
> E-gards: Jimmy
> _______________________________________________
> Devel mailing list
> Devel at lists.geany.org
> https://lists.geany.org/cgi-bin/mailman/listinfo/devel